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Final  Report:  Using  Game  Play  to  Diagnose  and  Remediate  Students’  Misconceptions  in  Solving  Equations 

ABSTRACT 

During  the  reporting  period  we  focused  on  conducting  an  error  analysis  of  students’  game  play  data  and  redesigning  the  game  levels  based 
on  the  types  of  errors  students  were  making.  The  error  analysis  examined  students’  step-by-step  process  of  simplifying  exponent 
expressions.  The  majority  of  errors  were  related  to  the  misapplication  of  the  power  rule,  incorrect  transformation  between  exponent  and  root 
forms,  apparent  confusion  involving  the  negative  sign  and  expressions  raised  to  a  power,  and  the  misapplication  of  the  negative  exponent 
rule.  These  errors  were  the  basis  for  redesigning  the  game  levels  and  assessments. 

The  new  game  level  design  focuses  on  (a)  exponent  to  root  transformations,  (b)  the  negative  exponent  rule,  and  (c)  the  power  rule.  Sets  of 
game  levels  have  been  designed  to  structure  game  play  to  emphasize  varied  practice  and  reflection.  The  revised  design  also  includes  a  self¬ 
explanation  prompt  to  emphasize  reflection  and  a  challenge  problem  that  gives  students  only  a  single  chance  to  solve  the  problem  (which 
also  serves  as  an  in-game  measure  of  learning). 

Next  steps  include  implementing  the  revised  game  design  and  updating  the  pretest  and  post-test  to  match  the  game  levels,  gathering  data  on 
the  revised  game,  and  publishing  the  work  in  a  book  chapter. 
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The  focus  of  this  project  was  to  develop  and  test  the  effectiveness  of  gameplay  on  learning  of  solving  equations  concepts 
with  students  enrolled  in  a  developmental  math  course  at  California  State  University,  San  Bernardino  (CSUSB).  The  project 
combines  the  subject  matter  expertise  of  the  math  department  at  CSUSB,  the  game  development  and  assessment  expertise  of 
CRESST,  and  the  expertise  on  motivational  issues  (particularly  with  the  Hispanic/Latino  students  at  CSUSB)  of  the  University  of 
Southern  California  (USC). 

Prior  research  on  the  solving  equations  suggests  that  simplifying  expressions  can  be  a  mysterious  process  to  students  who 
have  poor  understanding  of  mathematics  (Chung  and  Delacruz,  2014).  The  mathematical  justifications  for  a  transition  from  one 
step  to  the  next,  as  depicted  in  textbooks  and  typical  math  instruction,  often  focuses  only  on  the  major  transformative  operations 
(e.g.,  distribution,  identity  properties)  and  often  ignores  the  steps  that  may  be  tedious  to  illustrate  and  too  obvious  to  include  (e. 
g.,  regrouping,  associative,  commutative).  Yet  for  students  not  fluid  in  the  interpretation  or  use  of  the  mathematical  rules  and 
properties,  these  intermediate  steps  as  well  as  the  more  general  technique  of  specifying  the  step-to-step  transition  may  be  a 
fruitful  way  to  support  the  development  of  students’  skill  in  simplifying  expressions.  The  game  developed  in  this  grant 
implements  a  design  that  uses  the  step-to-step  transitions  as  the  core  game  mechanic,  and  exposes  students  to  the  “micro 
steps”  often  left  out  of  textbook  and  classroom  examples. 

This  project  was  accomplished  over  four  phases.  The  first  phase  involved  identifying  the  specific  mathematical  topics  to  be 
tested.  Additionally  this  initial  phase  included  identifying  the  target  outcome  performance,  which  was  for  students  to  be  able  to 
work  through  a  problem  and  explain  their  problem  solving  process  step  by  step.  This  phase  took  place  from  May  2012  through 
July  2012. 

During  the  second  phase,  a  prototype  game  developed  and  pilot  tested  with  CSUSB  students.  Based  on  student  responses,  the 
prototype  was  updated  to  modify  the  type  of  feedback  provided  about  specific  errors.  This  modified  version  was  also  class 
tested  during  this  phase.  This  phase  took  place  from  August  2012  through  July  2013. 

During  the  third  phase,  we  focused  on  the  major  types  of  mathematical  errors  committed  by  students  during  game  play  and 
redesigning  the  game  to  provide  more  focused  practice  on  those  topics.  This  phase  took  place  from  August  2013  through  July 
2014. 

During  the  final  phase,  the  revised  game  was  class  tested.  An  analysis  of  the  results  was  completed. 


Phase  I 

Identify  the  math  domain  and  outcome  performance 

During  the  summer  of  2012  CSUSB  identified  the  math  domain,  course,  and  particular  topics  within  the  course  to  target  for 
gameplay.  Math  90,  Development  Mathematics  (intermediate  algebra),  was  the  course  selected  for  this  project.  The  major 
topics  in  Math  90  are:  (a)  Systems  of  Linear  Equations  and  Inequalities;  (b)  Equations  and  Inequalities  in  One  Variable;  (c) 
Graphs,  Relations,  and  Functions;  (d)  Introduction  to  Graphing  and  Equations  of  Lines;  (e)  Rational  Expressions  and  Equations; 
(f)  Radicals  and  Rational  Exponents;  and  (g)  Quadratic  Equations  and  Functions. 

Within  these  topics,  the  domain  Radicals  and  Rational  Exponents  was  selected.  The  sub-topics  that  will  be  tested  are:  nth  roots 
and  rational  exponents  and  simplify  expressions  using  laws  of  exponents.  These  sub-topics  were  selected  because  of  their 
difficulty  for  a  large  number  of  students  and  their  importance  to  subsequent  mathematics.  A  poor  understanding  of  the  laws  of 
exponents  will  not  only  result  in  a  poor  understanding  of  operations  on  exponents,  but  also  impoverished  understanding  of 
advanced  topics  that  require  exponents  such  as  quadratic  equations  and  their  applications. 

Target  performance:  The  target  outcome  performance  is  for  students  to  be  able  to  work  through  a  problem  and  explain  their 
problem  solving  process  step-to-step. 

Identify  the  common  errors  that  students  possess  in  that  domain 

CSUSB  and  CRESST  have  conducted  several  knowledge  acquisition  sessions  that  have  focused  on  identifying  the  common 
student  errors  with  respect  to  exponents.  The  purpose  for  focusing  on  common  errors  is  that  common  errors  provide  a  glimpse 
into  (the  lack  of)  student  understanding — the  prior  knowledge  students  may  lack  that  lead  to  such  errors  and  the  kind  of  thinking 
processes  and  aberrant  beliefs  students  hold  that  may  cause  them  to  approach  problem  solving  in  a  particular  way. 


Common  errors  also  provide  convenient  “traps”  that  in  a  game  context  can  result  in  engaging  gameplay.  This  particular  aspect 
of  gameplay — making  play  challenging — can  both  expose  students’  brittle  understanding  and  provide  instructional  opportunities 
via  instructional  feedback  and  player-initiated  help.  The  idea  of  impasse-driven  learning  and  how  to  best  structure  instructional 
feedback  in  a  game  is  a  major  long-term  scientific  goal  of  the  project.  Examples  of  the  kinds  of  common  errors  fell  into  six 
categories  and  are  shown  in  Appendix  PI -A  (See  the  attachment). 

Identify  candidate  game  mechanics  to  test  that  could  be  used  for  instructional  and  assessment  purposes 

A  clear  requirement  for  the  game  that  flowed  from  the  knowledge  acquisition  sessions  is  that  students  often  simply  memorize 
procedures  and  do  not  understand  why  a  certain  mathematical  operation  applies — “it  simply  does.”  Thus,  the  CSUSB  SME 
believed  it  was  very  important  that  the  game  help  students  make  explicit  their  thinking  process  via  interrogation  techniques  (e. 
g.,  posing  questions  to  students  and  not  simply  telling  students;  having  students  explain  what  they  are  doing  and  not  just 
require  them  to  do  something).  These  also  happen  to  be  techniques  used  in  games  to  increase  players’  interest  and 
engagement  in  games. 

The  candidate  game  mechanics  are: 

1 .  Questioning  techniques.  For  example,  an  avatar  asks  the  student  whether  he  or  she  thinks  an  operation  might  work  or  elicits 
from  students  the  next  step  in  the  problem  solving  process. 

2.  Explanation  techniques.  For  example,  require  the  student  to  explain  why  he  or  she  thinks  a  particular  mathematical  operation 
might  work.  Or  require  students  to  explain  the  critical  difference  between  two  contrasting  cases. 

3.  Use  of  worked  examples.  For  example,  provide  a  worked  example  of  how  to  solve  an  equation  and  slowly  fade  away  the 
amount  of  support  over  a  series  of  problems. 

Develop  measures 

A  set  of  measures  have  been  identified  for  the  research.  The  types  of  measures  are:  (a)  prior  knowledge  of  the  content;  (b) 
post-game  knowledge  of  the  content;  (c)  background  information  such  as  prior  math  grades,  SAT/ACT  scores,  game 
experience,  demographics  information  (age,  sex,  race,  class  standing,  major);  (d)  English  language  proficiency;  and  (e)  math 
self-efficacy  and  interest  in  math.  Draft  measures  are  shown  in  Appendix  P1-B  to  Appendix  P1-E  (See  the  attachment). 

The  knowledge  measures  will  target  the  specific  content  covered  in  the  game  in  various  formats  (e.g.,  cast  in  terms  of  the 
game;  cast  symbolically  as  in  a  typical  math  test).  Items  susceptible  to  common  errors  also  will  be  included  in  the  item  set,  as 
well  as  items  that  require  students  to  explain  their  problem  solving  step-by-step.  The  non-knowledge  items  will  be  adapted  from 
prior  studies. 

Develop  high-level  game  requirements 

A  preliminary  set  of  broad  requirements  for  the  game  was  developed  from  the  knowledge  acquisition  sessions.  Specifically,  the 
following  strategies  were  identified  as  desirable  for  the  Math  90  student  population: 

Use  interrogation  strategies  typical  in  games — questioning  avatar  (vs.  telling  avatar) 

Embed  motivational  components  such  as  incremental  success  and  self-reflection  strategies 

Focus  on  diagnostics — detect  what  students  know  and  common  errors 

Report  diagnostics  back  to  instructor  and  students 

Focus  on  type  of  instructional  feedback 

Questioning  vs.  telling — good  game  design 

Challenge  level — ok  for  “weird”  assessment  levels 

Focus  directly  on  symbolic  manipulation 

Phase  2 

During  this  phase  a  prototype  game  was  developed  and  pilot  tested  with  CSUSB  students.  Based  on  student  responses,  the 
prototype  was  updated  to  modify  the  type  of  feedback  provided  about  specific  errors.  This  modified  version  was  also  class 
tested  during  this  phase. 

The  prototype  game  consisted  of  22  levels.  See  Appendix  P2-A  (See  the  attachment)  for  a  screenshot  of  the  prototype  version 
of  the  game.  Levels  1-12  required  participants  to  select  the  immediate  next  step  to  the  problem.  Levels  13-22  required 
participants  to  solve  multistep  problems.  Participants  were  shown  a  total  points  (one  point  for  each  correct  step  selected), 
bonus  points  (one  point  for  each  “mental  step”  step  elected),  and  total  completed  problems  (one  point  for  each  problem 
completed  before  time  ran  out). 


Based  on  the  pilot  test  performed  in  November  2012,  it  was  apparent  that  students  not  only  enjoyed  playing  the  game,  but  also 


felt  that  they  learned  from  the  game.  In  particular,  students  reported  that  they  thought  that  receiving  feedback  immediately  on 
each  mathematical  step  was  more  effective  than  having  to  work  through  an  entire  math  problem  and  not  know  which  step  in  the 
process  might  have  contributed  to  an  incorrect  solution. 

However,  student  responses  related  to  the  next  step  process  (as  a  form  of  instruction)  and  the  type  of  feedback  provided  (e.g., 
receiving  feedback  about  specific  errors)  seemed  somewhat  unclear.  As  such  the  prototype  was  updated  to  provide  more 
explicit  and  targeted  feedback  options  related  to  each  mathematical  step,  and  more  sophisticated  help  options/menus  for 
students  to  access  when  experiencing  difficulty  during  gameplay.  Screenshots  depicting  example  feedback  options  and  help 
menus  are  provided  in  Appendix  P2-B  (See  the  attachment). 

Additional  levels  were  also  created  to  reflect  more  consistency  across  math  item  expressions  and  steps.  In  other  words, 
representations  of  mathematical  expressions  or  transformations  were  created  to  be  consistent  across  items  as  well  as 
consistent  with  how  the  students  were  taught  to  transform  math  items  and  depict  math  expressions.  The  following  section 
provides  further  detail  about  the  specific  instructional  variations  within  the  game. 

Develop  instructional  variations  within  the  game 

The  following  key  instructional  variation  components  were  developed  for  testing  purposes  within  the  game: 

Step-to-step  transitions:  the  math  problems  were  structured  such  that  students  were  able  to  observe  the  steps  required  to 
solve  the  problem  including  incorrect  steps  and  common  errors,  and  select  from  those  steps  the  next  correct  step  in  the 
problem  sequence.  See  Appendix  P2-C  for  an  example  (See  the  attachment). 

Help  menu  with  worked  examples  and  explanations:  these  help  features  were  available  for  students  to  access  when 
needed  during  gameplay,  and  certain  feedback  prompts  recommended  students  view  specified  help  menus  for  further 
explanation.  (See  Appendix  P2-B  in  the  attachment) 

Feedback  variations 

Implicit:  this  type  of  feedback  allowed  students  to  understand  that  the  step  they  selected  was  either  incorrect  or  correct  by 
either  sticking  to  or  not  sticking  to  the  target  problem  or  preceding  step. 

Knowledge  of  results:  this  type  of  feedback  provided  students  with  an  image  or  statement  indicating  that  their  action  was 
correct  or  incorrect  (e.g.,  “Sorry,  that  was  incorrect.  Please  try  again.”),  or  that  they  received  a  bonus  point. 

Elaboration:  this  type  of  feedback  provided  students  with  an  elaborated  statement  and  suggestion  about  their  miss  step  (e. 
g.,  “Oops!  Looks  like  you  separated  the  negative  and  you  shouldn’t  have.  It  should  stay  with  the  125  under  the  root  bar.  Think 
about  what  number  is  the  cube  root  of  125”).  A  variation  of  this  feedback  type  also  included  an  image  of  an  example  problem, 
or  directing  students  to  the  help  menu  for  further  examples. 


Determine  overall  game  effectiveness  and  effectiveness  of  instructional  variations 

To  determine  overall  game  effectiveness  and  usability,  and  effectiveness  of  instructional  variations,  three  tests  were  performed; 
November  2012,  February  2013,  and  May  2013.  The  below  sections  detail  the  November  2012  and  February  201 3  studies. 

The  data  for  the  May  2013  study  are  being  analyzed. 

November  2012:  The  purpose  of  the  November  2012  testing  was  to  determine  game  usability,  receive  student  feedback  on  the 
game-playing  experience,  and  test  the  software  data  logging.  Fifteen  participants  played  the  prototype  game  for  30-60  minutes. 
After  playing  the  prototype  game,  participants  were  administered  a  survey.  The  survey  consisted  of  24  selected  response 
questions  and  eight  open-ended  response  questions. 

Participants  reported  enjoying  the  experience  of  playing  the  prototype.  Participants  also  reported  that  they  learned  from  the 
game  and  that  the  game  helped  them  with  something  they  didn’t  know  before.  They  thought  that  this  type  of  activity  (getting 
feedback  immediately  on  each  step)  was  more  effective  than  other  similar  activities  such  as  homework.  However,  when  asked 
specifically  about  the  next  step  process  and  receiving  feedback  about  errors,  there  was  a  wide  range  of  responses  They  also 
thought  that  earning  points,  especially  the  bonus  points,  was  motivating. 

February  2013:  The  purpose  of  the  February  2013  testing  was  to  determine  game  effectiveness  and  effectiveness  of  measures, 
check  for  learning  gains  and  condition  differences  (i.e.,  feedback  variations),  gather  student  perception  data  (i.e.,  math  attitudes 
and  game  perceptions),  and  determine  that  student  process  data  was  being  logged  correctly  during  gameplay. 

Participants  were  randomly  assigned  to  both  the  control  (n=  57)  and  treatment  (n=  54)  condition.  Data  analysis  indicated  that 
the  conditions  were  similar  across  most  background  and  demographic  variables.  However,  the  control  group  reported 
significantly  higher  (p  =  .04)  interest  in  math  than  the  treatment  group,  and  marginally  significant  differences  (p  =  .06)  were 
found  for  GPA  between  groups  with  the  treatment  group  scoring  higher.  No  differences  were  found  in  pretest  scores  and  self¬ 
belief  in  math.  (See  Appendix  P2-D,  Table  1  for  details  in  the  Attachment). 

An  ANOVA  was  conducted  to  determine  if  there  was  an  interaction  between  assessment  form  and  condition.  No  significant 
interaction  was  found  on  pretest  or  posttest,  and  no  main  effects  for  assessment  form  or  condition  were  found  on  pretest  or 
posttest.  Results  further  indicated  that  were  was  no  significant  difference  between  conditions  on  the  posttest. 

Analysis  of  performance  on  math  scores  revealed  that  there  was  an  approximately  1 1  %  gain  from  pretest  to  posttest  for  both 
the  control  and  treatment  condition.  (See  Appendix  P2-D,  Table  2  for  details)  (See  the  attachment).  Table  3  (see  Appendix  P2- 
D  in  the  Attachment)  reports  item-level  performance  on  the  pretest  and  posttest.  These  findings  indicated  that  approximately 
12%  of  students  scored  incorrect  on  two  pretest  items  then  correct  on  the  posttest,  and  18%  increased  from  incorrect  to  correct 
on  and  additional  three  other  items  on  the  posttest.  The  remaining  two  items  showed  increases  between  28%  and  40%  of 


students  from  pretest  to  posttest.  Individual  test  items  were  also  analyzed  using  the  McNemar  test  to  determine  significant 
change  from  pretest  to  posttest.  Results  indicated  a  significant  change  from  pretest  to  posttest  for  items  5,  10,  and  1 1 .  (see 
Table  3,  Appendix  P2-D  in  the  Attachment).  In  sum,  the  results  indicated  a  small  overall  increase  of  scores  from  pretest  to 
posttest,  with  higher  performance  gains  appearing  for  specific  items. 

Based  on  the  February  2013  testing,  a  few  more  levels  were  added  to  the  game  to  target  additional  math  problems,  some 
images  were  changed  to  more  clearly  depict  feedback  and  examples,  and  some  items  on  the  pre  and  post  assessment  were 
changed  based  on  instructor  feedback  and  student  performance. 

May  2013:  The  main  purpose  of  the  May  2013  testing  was  to  evaluate  an  initial  version  of  the  game  based  on  changes  after  the 
February  results.  Similar  to  February  2013  testing,  the  goal  for  May  2013  testing  was  also  to  determine  game  effectiveness  and 
learning  gains,  check  for  differences  across  conditions  (i.e.,  feedback,  no  feedback),  and  test  software  data  logging  and  gather 
student  perception  data  regarding  math  and  gameplay. 

Phase  3 

The  major  scientific  objectives  of  this  phase  were  to  determine  the  major  types  of  mathematical  errors  committed  by  students 
during  gameplay  and  to  redesign  the  game  levels  to  provide  more  focused  practice  on  those  topics. 

Error  Analysis 

Prior  to  conducting  the  error  analysis,  the  problems  used  in  the  game  were  subjected  to  a  feature  analysis.  The  feature  analysis 
identified  unique  properties  of  the  problems  in  terms  of  the  structure  of  the  problem  (e.g.,  whether  the  problem  has  negative 
terms),  cognitive  demands  (e.g.,  whether  students  were  required  to  transform  an  expression  from  an  exponent  form  to  a  radical 
form),  and  mathematical  topic  (e.g.,  application  of  the  negative  exponent  rule).  Table  1  (See  the  attachment)  shows  the  set  of 
features  used  to  describe  the  problems  used  in  the  game. 

Types  of  Errors 

The  problem  steps  associated  with  errors  committed  by  at  least  30  students  were  examined  and  represented  25  different  errors. 
Thirty  students  represented  30%  of  the  sample  and  the  types  of  errors  appeared  to  be  representative  of  the  remaining  errors  in 
the  sample.  The  errors  were  committed  over  12  game  levels,  with  level  45  having  the  highest  number  of  errors  (7). 

Errors  were  detected  when  students  attempted  to  specify  the  next  step  given  an  existing  step.  In  general,  the  next  step 
represented  the  application  (or  misapplication)  of  an  exponent  rule  and  inspection  of  the  errant  next  step  provided  insight  into 
students’  understanding  as  students  attempted  to  simplify  the  expression.  For  example,  on  level  21,  given  the  expression,  3A 
(1/3)-3A(1/3),  58  students  specified  9A(1/9)  as  the  next  step  and  57  students  specified  3A(1/9)  as  the  next  step.  In  both  cases, 
the  misapplication  of  the  power  rule  is  clear  and  reflects  a  common  error  of  multiplying  exponents  (instead  of  adding  them).  The 
second  error  also  reflects  an  additional  misconception  of  multiplying  the  base.  The  25  errors  committed  by  at  least  30  or  more 
students  is  given  in  Appendix  P3-A  (See  the  attachment)and  frequency  information  on  all  errors  committed  by  all  students  is 
given  in  Appendix  P3-B  (See  the  attachment). 

To  better  understand  the  context  of  the  errors,  each  error  was  examined  in  terms  of  the  concept  that  was  misapplied,  the  term 
property,  and  problem  type  (as  defined  by  the  feature  analysis  in  Table  1  (See  the  attachment)).  Many  of  the  errors  were  related 
to  the  misapplication  of  the  power  rule,  incorrect  transformation  between  exponent  and  root  forms,  apparent  confusion  about 
the  convention  related  to  the  negative  sign  and  powers,  and  the  misapplication  of  the  negative  exponent  rule.  These  major 
errors  served  as  the  basis  for  redesigning  the  game  levels  and  assessments. 

Game  Level  Revisions 

The  revised  game  level  design  incorporates  the  results  of  the  error  analysis  in  the  following  ways.  First,  the  problems  in  the  new 
game  levels  focus  exclusively  on  the  following  concepts:  (a)  exponent  to  root  transformations;  (b)  negative  exponent  rule;  and 
(c)  power  rule.  Second,  the  problems  incorporate  negative  numbers  both  as  exponent  properties  as  well  as  term  properties. 
Finally,  problems  sets  composed  of  4  levels  have  been  created  to  structure  gameplay  to  emphasize  practice  and  reflection.  As 
in  the  original  game,  the  game  levels  are  divided  into  2  stages.  Stage  1  is  designed  to  provide  students  with  practice  on  basic 
concepts  and  Stage  2  is  designed  to  provide  students  with  applying  the  concepts.  Additionally,  two  types  of  problem  types  have 
been  added:  a  self-explanation  question  that  asks  students  to  explain  their  choice,  and  a  challenge  problem  that  gives  students 
only  a  single  chance  to  transform  the  problem.  The  challenge  problem  serves  as  an  in-game  measure  of  learning  for  a 
particular  problem  set. 

Stage  1:  Single  step  transformations.  Transformations  cover  basic  concepts  using  single-term,  single-step  problems.  There  are 
six  problem  sets  (PS)  in  Stage  1 .  PS  1  requires  students  to  transform  from  exponential  form  to  root  form,  PS  2  requires 
students  to  transform  negative  exponents  into  positive  exponents,  and  PS  3  combines  these  two  concepts.  PS  4  provides 
practice  with  negative  signs.  PS  5  increases  complexity  of  expressions  by  including  negative  terms  and  positive  and  negative 
fractional  exponents.  PS  6  introduces  students  to  the  power  rule,  required  for  multi-term  problems  in  Stage  2. 

Each  PS  includes  four  problems:  (a)  a  single-step  problem  with  feedback;  (b)  a  single-step  problem  requiring  students  to 
explain  their  reasoning  behind  their  answer;  (c)  another  single-step  problem  with  feedback  to  gauge  any  change  in  their 
reasoning  based  on  their  self-explanation;  and  (d)  a  challenge  problem  with  no  feedback.  Table  2  (See  the  attachment)  shows 
the  Stage  1  problem  set  sequence. 

Stage  2:  Multistep  transformations.  Stage  2  problem  sets  require  multistep  solutions.  PS  7  includes  four  problems:  Problems  a 
to  c  are  multi-step  problems  with  feedback  and  problem  d  is  a  challenge  problem.  No  self-explanation  problems  are  included. 
The  remaining  problem  sets  require  some  use  of  the  power  rule  to  distribute  an  exponent  to  multiple  terms.  The  steps  also 


require  students  to  apply  the  rules  learned  in  the  Stage  1  problem  sets.  PS  8  to  11  include  three  problems:  Problems  a  and  b 
are  multi-step  problems  with  feedback  and  problem  c  is  a  challenge  problem.  No  self-explanation  problems  are  included.  Table 
3  (See  the  attachment)  shows  the  Stage  2  problem  set  sequence. 

Phase  4 

Game  goal  and  mechanics.  The  goal  of  the  game  is  for  players  to  simplify  a  given  expression  by  specifying  the  next  step  in  the 
simplification  process.  To  start  the  process,  a  single  tile  with  a  picture  of  the  to-be-simplified  expression  is  placed  on  the  main 
game  board.  Time  pressure,  a  common  game  design  element,  is  incorporated  by  having  the  tile  slowly  creep  to  the  top  of  the 
game  board.  The  player  must  select  the  “next  step”  from  a  set  of  available  tiles  on  left  side  of  the  game  board  and  drop  it  on  the 
target  tile  as  shown  in  Figure  1  (See  the  attachment).  If  the  dropped  tile  is  a  correct  next  step,  the  dropped  tile  is  attached  to  the 
bottom  of  the  target  tile.  In  Figure  1,  the  initial  tile  is  the  expression  (-125)A(1/3)  and  a  valid  next  step  has  been  dropped  on  the 
initial  tile.  The  stack  is  slowly  moving  to  the  top  of  the  board. 

To  help  students  to  focus  on  the  micro  steps  between  the  transitions  between  major  steps,  the  game  awards  bonus  points  when 
students  use  a  micro  step,  with  a  “bonus”  splash  screen  appearing  momentarily  to  indicate  a  bonus  (or  use  of  a  micro  step). 
Instructional  design.  The  game  focuses  on  the  following  concepts:  (a)  exponent  to  root  transformations;  (b)  negative  exponent 
rule;  and  (c)  power  rule.  Many  of  the  problems  incorporate  negative  numbers  both  as  exponent  properties  as  well  as  term 
properties.  Problem  sets  composed  of  4  levels  each  structure  gameplay  to  emphasize  practice  and  reflection.  The  game  levels 
are  divided  into  2  stages.  Stage  1  is  designed  to  provide  students  with  practice  on  basic  concepts  and  Stage  2  is  designed  to 
provide  students  with  applying  the  concepts.  Additionally,  challenge  problems  exists  that  gives  students  only  a  single  chance  to 
transform  a  problem.  The  challenge  problem  also  serves  as  an  in-game  measure  of  learning. 

Stage  1  design.  There  were  24  levels  in  Stage  1.  The  levels  cover  basic  concepts  using  single-term,  single-step  problems. 

There  are  six  problem  sets  (PS)  in  Stage  1 .  PS  1  requires  students  to  transform  from  exponential  form  to  root  form,  PS  2 
requires  students  to  transform  negative  exponents  into  positive  exponents,  and  PS  3  combines  these  two  concepts.  PS  4 
provides  practice  with  negative  signs.  PS  5  increases  complexity  of  expressions  by  including  negative  terms  and  positive  and 
negative  fractional  exponents.  PS  6  introduces  students  to  the  power  rule,  required  for  multi-term  problems  in  Stage  2. 

Each  problem  set  includes  four  problems:  (a)  a  single-step  problem  with  feedback;  (b)  a  single-step  problem  requiring  students 
to  explain  their  reasoning  behind  their  answer;  (c)  another  single-step  problem  with  feedback  to  gauge  any  change  in  their 
reasoning  based  on  their  self-explanation;  and  (d)  a  challenge  problem  with  no  feedback.  Table  4  (See  the  attachment)  shows 
Stage  2  design.  There  are  16  levels  in  Stage  2.  The  levels  in  Stage  2  problem  sets  require  multistep  solutions.  PS  7  includes 
four  problems:  Problems  a  to  c  are  multi-step  problems  with  feedback.  No  self-explanation  problems  are  included  in  Stage  2. 
The  remaining  problem  sets  require  some  use  of  the  power  rule  to  distribute  an  exponent  to  multiple  terms.  The  steps  also 
require  students  to  apply  the  rules  learned  in  the  Stage  1  problem  sets.  PS  8  to  11  include  three  problems:  Problems  a  and  b 
are  multi-step  problems  with  feedback  and  problem  c  is  a  challenge  problem.  No  self-explanation  problems  are  included.  Table 
3  (See  the  attachment)  shows  the  Stage  2  problem  set  sequence. 


Research  Questions 

The  main  research  questions  for  this  study  are:  To  what  extent  does  self-explanation  of  exponent  rules  in  a  game  affect  their 
application  of  exponent  rules  and  perceptions  of  engagement?  To  what  extent  does  can  game-based  practice  of  exponent  rules 
improve  students’  application  of  those  exponent  rules? 

Method 

Participants  and  Design 

Participants.  The  sample  comprised  69  participants  (19  males,  47  females,  3  unreported)  who  were  ethnically  diverse  (42 
Latino/a,  9  White,  8  Black  or  African  American,  2  Asian  or  Pacific  Islander,  3  Multiracial/multiethnic,  and  5  unspecified)  and  with 
a  mean  age  of  19.2  years  old  (SD  =  3.8  years).  When  asked  how  frequently  a  language  other  than  English  is  spoken  at  home, 
20  students  (29%)  reported  all  or  most  of  the  time,  23  (33%)  about  half  of  the  time,  12  (17%)  once  in  a  while,  and  1 1  (16%) 
never.  Eighteen  students  reported  they  were  repeating  the  class,  28  students  took  the  preceding  course,  and  20  students 
reported  this  was  their  first  college  math  course.  The  mean  overall  GPA  reported  by  students  was  3.16  (SD  =  0.47)  on  a  4.0 
scale.  When  asked  about  their  gameplay  skill,  16  students  (23%)  reported  they  were  very  good,  13  (19%)  reported  good,  24 
(35%)  reported  their  skill  as  fair,  and  13  (19%)  reported  poor  gameplay  skills.  When  asked  how  many  hours  per  week  they 
spend  playing  games,  26  participants  (38%)  reported  0  hours,  29  participants  (42%)  reported  1-4  hours,  7  participants  reported 
5-8  hours,  and  1  participant  report  13  or  more  hours.  Finally,  the  mean  score  (on  a  1  to  4  point  scale)  for  participants’  interest  in 
math  and  math  self-efficacy  were  M  =  2.48  (SD  =  0.71)  and  M  =  2.5  (SD  =  0.75).  Overall,  the  sample  appeared  to  be  of  average 
achievement,  ethnically  diverse,  and  have  a  moderate  degree  of  game  experience.  Participants’  interest  and  self-efficacy  were 
at  the  scale  midpoints  and  normally  distributed,  suggesting  the  sample  was  not  overly  interested  or  disinterested  in  math,  nor 
excessively  possess  high  or  low  in  math  self-efficacy. 

Design.  A  two-group,  pretest-posttest  experimental  design  was  used.  Participants  were  randomly  assigned  to  either  a  baseline 
game  condition  or  a  self-explanation  condition. 

Measures 

Prerequisite  knowledge.  Four  questions  were  asked  to  gather  information  on  participants’  proficiency  with  basic  fraction 
addition,  multiplication,  and  order  of  operations. 

Rules  of  exponent.  Three  scales  were  developed  to  measure  three  skills:  (a)  converting  an  exponent  to  its  root  form  (ETR),  (b) 


application  of  the  negative  exponent  rule  (NER),  and  (c)  application  of  the  power  rule  (PR).  A  fourth  scale  was  developed  to 
examine  transfer  performance  (XFER)  and  included  topics  from  each  of  the  other  scales  but  the  problems  were  more  complex 
in  terms  of  the  number  of  variables  in  the  expressoin.  The  ETR  scale  had  8  items  (pretest  a  =  .73;  posttest  a  =  .74),  NER  scale 
6  items  (pretest  a  =  .60;  posttest  a  =  .73),  the  PR  scale  had  4  items  (pretest  a  =  .82;  posttest  a  =  .65),  and  the  XFER  scale  had 
5  items  (posttest-only  a  =  .71). 

Game  perception.  Two  scales  were  used  to  measure  participants’  perception  of  their  game  experience.  The  engagement  scale 
(I  really  got  into  the  game,  Doing  the  game  was  boring,  I  would  have  liked  to  play  longer,  I  thought  the  game  was  a  waste  of 
time,  Beating  the  different  levels  made  me  feel  good)  and  a  perceived  learning  scale  (I  learned  something  new  about 
exponents/roots/math  from  the  game,  The  game  helped  me  figure  out  something  I  previously  had  trouble  with,  I  realized  I  had  a 
misconception  about  an  exponent/root  topic  while  playing  the  game).  The  5-item  perceived  engagement  scale  a  =  .75  and  3- 
item  perceived  learning  scale  a  =  .85. 

Feedback  about  the  game.  Participants  were  asked  to  provide  feedback  about  their  experience  with  the  game.  Participants 
were  asked  three  questions  (How  do  you  think  this  game  compares  to  other  ways  of  practicing  math  (e.g.,  doing  problems  on 
paper)?;  How  helpful  (for  learning)  do  you  think  it  was  to  emphasize  the  “next  step”  in  the  problem  solving  process?  Overall, 
how  much  did  you  learn  about  exponents/roots  from  the  game?).  Each  question  had  Likert-type  anchors  as  well  as  space  for 
written  comments. 

Background.  Participants  were  asked  to  complete  a  questionnaire  asking  for  their  age,  gender,  ethnicity,  and  frequency  of 
speaking  a  language  other  than  English  at  home.  In  addition,  participants  self-reported  their  overall  GPA,  transfer  status,  skill 
with  video  games,  and  number  of  hours  per  week  spent  playing  games.  Participants  were  also  asked  about  their  interest  in 
math  and  math  self-efficacy.  These  scales  were  adopted  from  Marsh,  Hau,  Artlet,  Baumert,  and  Peschar  (2006).  The  interest  in 
math  scale  had  3  items  (a  =  .72)  and  math  self-efficacy  scale  3  items  (a  =  .80). 

Game 

Base  version.  The  base  version  was  the  game  as  described  earlier.  There  were  24  levels  requiring  single-step  transformations 
(Stage  1)  and  16  levels  require  multiple-step  transformations  (Stage  2).  Students  were  not  required  to  self-explain  their 
selections. 

Experimental  version.  The  experimental  version  of  the  game  required  participants  to  engage  in  self-explanation  during  selected 
levels  in  Stage  1.  There  were  6  levels  that  required  self-explanation.  There  were  4  levels  in  a  problem  set.  The  design  of  the 
levels  within  a  problem  set  was  to  first  provide  a  level  that  provided  minimal  feedback  to  participants  if  they  selected  an 
incorrect  step.  Then  a  level  with  a  similar  problem  was  presented  except  with  a  self-explanation  prompt  appearing  immediately 
after  the  student  dropped  the  next  step  tile  onto  target  tile  on  the  game  board.  No  feedback  was  given  to  students  about  the 
accuracy  of  their  self-explanation.  The  self-explanation  prompt  was  a  list  of  exponent  rules.  Participants  were  instructed  to 
choose  the  rule  that  justified  their  next  step  as  shown  in  Figure  2  (See  the  attachment). 

Procedure 

Participants  were  introduced  to  the  study  and  then  provided  all  materials  (pretest,  laptop  computer,  and  posttest).  Participants 
were  instructed  to  start  on  the  pretest.  After  completing  the  pretest,  the  pretest  was  collected  and  participants  were  instructed  to 
start  the  game  on  the  laptop.  After  completing  the  game  at  their  own  pace,  participants  started  the  posttest  and  completed  the 
test  at  their  own  pace.  All  participants  except  2  finished  all  materials.  These  two  participants  did  not  finish  the  game  nor  the 
posttest. 

Results 

Preliminary  Analyses 

A  check  for  differences  between  conditions  was  conducted  for  the  pretest  and  background  measures.  An  independent  t-test 
was  conducted  on  the  prerequisite  knowledge  items,  the  three  rules  of  exponent  scales,  the  two  attitudinal  scales,  and  the  two 
game  experience  questions.  There  were  no  differences  between  conditions  on  any  measure  (p  >  .05)  suggesting  that  the 
randomization  procedure  worked. 

Descriptive  Statistics 
(See  the  attachment) 

Main  Analyses 

Performance.  Separate  repeated-measures  ANOVAs  were  conducted  on  the  ETR,  NER,  and  PR  scales.  To  simultaneously 
determine  if  there  was  a  condition  effect  as  well  as  a  learning  effect,  participants  performance  were  analyzed  with  a  2(condition) 
c  2(occasion)  ANOVA,  with  occasion  as  the  within-subjects  factors,  and  condition  as  the  between-subjects  factor.  No  significant 
condition  c  occasion  interaction  was  found  for  ETR,  NER,  or  PR  scales,  suggesting  that  there  were  no  effects  of  self¬ 
explanation  on  participants’  learning  of  those  topics.  For  the  transfer  task,  an  independent  t-test  was  conducted  on  the  XFER 
scale  to  test  for  differences  between  conditions.  No  differences  were  found. 

However,  a  significant  within-subjects  main  effect  of  occasion  was  found  for  ETR  (F[1 , 65]  =  12.82,  MSE  =  1 .12,  p  =  .001),  NER 
(F[1, 65]  =  19.42,  MSE  =  17.16,  p  <  .001),  and  PR  (F[1, 65]  =  12.82,  MSE  =  14.13,  p  =  .001),  suggesting  that  participants 
learned  over  the  course  of  the  game.  Table  7  displays  the  descriptive  statistics,  effect  size  of  the  change  (Dunlap,  Cortina, 
Vaslow,  &  Burke,  1996),  and  percent  increase  for  each  scale.  The  largest  increase  was  found  for  the  exponent  to  root 
transformation  problems. 

Perceived  learning  and  engagement.  To  examine  the  effects  of  self-explanation  on  participants’  perceived  engagement,  an 
independent  t-test  was  conducted  to  test  the  effects  of  condition  on  perceived  learning  and  participants’  perceived  engagement. 


No  differences  were  found  for  participants’  perceived  learning.  However,  participants  who  received  the  self-explanation  prompts 
(M  =  3.07  SD  =  0.56),  compared  to  participants  who  did  not  receive  self-explanation  prompts  (M  =  2.68,  SD  =  0.58),  had 
significantly  lower  ratings  of  their  experience,  t(63)  =  2.73,  p  =  .008,  d  =  0.68.  The  large  effect  size  suggests  that  the  self¬ 
explanation  prompts  interfered  with  participants’  perceived  engagement  with  the  game. 

Usability.  The  last  analysis  examined  participants’  perceptions  of  their  experience.  Table  8  shows  the  frequency  of  responses  to 
three  questions:  (a)  How  do  you  think  this  game  compares  to  other  ways  of  practicing  math  (e.g.,  doing  problems  on  paper)?  (b) 
How  helpful  (for  learning)  do  you  think  it  was  to  emphasize  the  “next  step”  in  the  problem  solving  process?  and  (c)  Overall,  how 
much  did  you  learn  about  exponents/roots  from  the  game? 

Overall,  participants  reported  very  favorable  responses  to  the  game,  with  59%  of  the  same  reporting  the  game  was  more  or 
much  more  effective  than  other  ways  of  practicing  math.  This  is  remarkable  considering  the  game  was  in  a  pre-prototype  stage, 
did  not  mask  the  mathematics,  and  provided  minimal  game  garnishes.  Thirty-three  percent  of  students  reported  the  emphasis  of 
the  next  step  as  being  very  or  extremely  helpful  for  learning,  with  an  additional  39%  reporting  the  emphasis  on  “next  steps”  as 
helpful.  Finally,  27%  of  students  reported  the  game  as  helping  them  learn  a  lot  about  exponents/roots  and  67%  reporting  the 
game  help  them  some. 

Conclusion 

Overall,  the  pre-prototype  game  appeared  to  provide  students  with  an  engaging  and  beneficial  experience.  With  less  than  40 
minutes  of  gameplay  time,  students  improved  their  performance  on  math  problems  related  to  converting  an  exponent  to  its  root 
form,  applying  the  negative  exponent  rule,  and  applying  the  power  rule,  and  reported  high  levels  of  utility.  This  is  an  interesting 
finding  because  it  suggests  that  the  instructional  sequence,  game  mechanics,  and  focus  on  problems  susceptible  to  common 
errors  collectively  helped  students  improve  their  performance  substantially,  as  indicated  by  the  moderate  to  large  pretest-to- 
posttest  effects  sizes  ranging  from  .40  to  .73. 

The  lack  of  finding  an  effect  of  the  self-explanation  prompts  is  curious.  We  followed  the  similar  procedures  used  by  Mayer  and 
Johnson  (2010)  and  Johnson  and  Mayer  (2013),  who  found  that  the  when  people  are  asked  to  select  self-explanations  for  their 
moves  in  a  game,  they  learn  more  (compared  to  people  who  are  not  asked  to  select  an  explanation).  Our  results  are  more  in 
line  with  O’Neil  et  al.  (2014),  who  found  no  main  effect  of  self-explanation  in  math  learning  in  a  game.  O’Neil  et  al.  concluded 
that  the  most  effective  self-explanations  where  those  that  prompted  connections  between  the  concrete  elements  in  the  game 
and  the  abstract  mathematics  terminology.  The  least  effective  self-explanation  prompts  were  those  that  asked  very  simple  or 
very  abstract  questions.  It  may  be  the  case  for  our  game  and  participants,  the  self-explanation  prompts  may  been  too  abstract 
as  the  choices  were  the  symbolic  representation  of  the  exponent  rule.  Further  research  is  needed  to  explore  this  issue. 
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Using  Game  Play  to  Diagnose  and  Remediate  Students’ 
Misconceptions  in  Solving  Equations 
Contract  Number  W911NF-12-1-0077 
Final  Report 

The  focus  of  this  project  was  to  develop  and  test  the  effectiveness  of  gameplay  on  learning  of 
solving  equations  concepts  with  students  enrolled  in  a  developmental  math  course  at  California 
State  University,  San  Bernardino  (CSUSB).  The  project  combines  the  subject  matter  expertise  of 
the  math  department  at  CSUSB,  the  game  development  and  assessment  expertise  of  CRESST, 
and  the  expertise  on  motivational  issues  (particularly  with  the  Hispanic/Latino  students  at 
CSUSB)  of  the  University  of  Southern  California  (USC). 

Prior  research  on  the  solving  equations  suggests  that  simplifying  expressions  can  be  a 
mysterious  process  to  students  who  have  poor  understanding  of  mathematics  (Chung  and  Delacruz, 
2014).  The  mathematical  justifications  for  a  transition  from  one  step  to  the  next,  as  depicted  in 
textbooks  and  typical  math  instruction,  often  focuses  only  on  the  major  transformative  operations 
(e.g.,  distribution,  identity  properties)  and  often  ignores  the  steps  that  may  be  tedious  to  illustrate 
and  too  obvious  to  include  (e.g.,  regrouping,  associative,  commutative).  Yet  for  students  not  fluid 
in  the  interpretation  or  use  of  the  mathematical  rules  and  properties,  these  intermediate  steps  as 
well  as  the  more  general  technique  of  specifying  the  step-to-step  transition  may  be  a  fruitful  way 
to  support  the  development  of  students’  skill  in  simplifying  expressions.  The  game  developed  in 
this  grant  implements  a  design  that  uses  the  step-to-step  transitions  as  the  core  game  mechanic, 
and  exposes  students  to  the  “micro  steps”  often  left  out  of  textbook  and  classroom  examples. 

This  project  was  accomplished  over  four  phases.  The  first  phase  involved  identifying  the  specific 
mathematical  topics  to  be  tested.  Additionally  this  initial  phase  included  identifying  the  target 
outcome  performance,  which  was  for  students  to  be  able  to  work  through  a  problem  and  explain 
their  problem  solving  process  step  by  step.  This  phase  took  place  from  May  2012  through  July 
2012. 

During  the  second  phase,  a  prototype  game  developed  and  pilot  tested  with  CSUSB  students. 
Based  on  student  responses,  the  prototype  was  updated  to  modify  the  type  of  feedback  provided 
about  specific  errors.  This  modified  version  was  also  class  tested  during  this  phase.  This  phase 
took  place  from  August  2012  through  July  2013. 

During  the  third  phase,  we  focused  on  the  major  types  of  mathematical  errors  committed  by 
students  during  game  play  and  redesigning  the  game  to  provide  more  focused  practice  on  those 
topics.  This  phase  took  place  from  August  2013  through  July  2014. 


During  the  final  phase,  the  revised  game  was  class  tested.  An  analysis  of  the  results  was 
completed. 


Phase  I 

Identify  the  math  domain  and  outcome  performance 


During  the  summer  of  2012  CSUSB  identified  the  math  domain,  course,  and  particular  topics 
within  the  course  to  target  for  gameplay.  Math  90,  Development  Mathematics  (intermediate 
algebra),  was  the  course  selected  for  this  project.  The  major  topics  in  Math  90  are:  (a)  Systems  of 
Linear  Equations  and  Inequalities;  (b)  Equations  and  Inequalities  in  One  Variable;  (c)  Graphs, 
Relations,  and  Functions;  (d)  Introduction  to  Graphing  and  Equations  of  Lines;  (e)  Rational 
Expressions  and  Equations;  (f)  Radicals  and  Rational  Exponents;  and  (g)  Quadratic  Equations 
and  Functions. 

Within  these  topics,  the  domain  Radicals  and  Rational  Exponents  was  selected.  The  sub-topics 
that  will  be  tested  are:  nth  roots  and  rational  exponents  and  simplify  expressions  using  laws  of 
exponents.  These  sub-topics  were  selected  because  of  their  difficulty  for  a  large  number  of 
students  and  their  importance  to  subsequent  mathematics.  A  poor  understanding  of  the  laws  of 
exponents  will  not  only  result  in  a  poor  understanding  of  operations  on  exponents,  but  also 
impoverished  understanding  of  advanced  topics  that  require  exponents  such  as  quadratic 
equations  and  their  applications. 

Target  performance:  The  target  outcome  performance  is  for  students  to  be  able  to  work  through  a 
problem  and  explain  their  problem  solving  process  step-to-step. 


Identify  the  common  errors  that  students  possess  in  that  domain 


CSUSB  and  CRESST  have  conducted  several  knowledge  acquisition  sessions  that  have  focused 
on  identifying  the  common  student  errors  with  respect  to  exponents.  The  purpose  for  focusing  on 
common  errors  is  that  common  errors  provide  a  glimpse  into  (the  lack  of)  student 
understanding — the  prior  knowledge  students  may  lack  that  lead  to  such  errors  and  the  kind  of 


thinking  processes  and  aberrant  beliefs  students  hold  that  may  cause  them  to  approach  problem 
solving  in  a  particular  way. 


Common  errors  also  provide  convenient  “traps”  that  in  a  game  context  can  result  in  engaging 
gameplay.  This  particular  aspect  of  gameplay — making  play  challenging — can  both  expose 
students’  brittle  understanding  and  provide  instructional  opportunities  via  instructional  feedback 
and  player-initiated  help.  The  idea  of  impasse-driven  learning  and  how  to  best  structure 
instructional  feedback  in  a  game  is  a  major  long-term  scientific  goal  of  the  project.  Examples  of 
the  kinds  of  common  errors  fell  into  six  categories  and  are  shown  in  Appendix  PI -A. 


Identify  candidate  game  mechanics  to  test  that  could  be  used  for  instructional  and 
assessment  purposes 


A  clear  requirement  for  the  game  that  flowed  from  the  knowledge  acquisition  sessions  is  that 
students  often  simply  memorize  procedures  and  do  not  understand  why  a  certain  mathematical 
operation  applies — “it  simply  does.”  Thus,  the  CSUSB  SME  believed  it  was  very  important  that 
the  game  help  students  make  explicit  their  thinking  process  via  interrogation  techniques  (e.g., 
posing  questions  to  students  and  not  simply  telling  students;  having  students  explain  what  they 
are  doing  and  not  just  require  them  to  do  something).  These  also  happen  to  be  techniques  used  in 
games  to  increase  players’  interest  and  engagement  in  games. 


The  candidate  game  mechanics  are: 

1.  Questioning  techniques.  For  example,  an  avatar  asks  the  student  whether  he  or  she  thinks  an 
operation  might  work  or  elicits  from  students  the  next  step  in  the  problem  solving  process. 

2.  Explanation  techniques.  For  example,  require  the  student  to  explain  why  he  or  she  thinks  a 
particular  mathematical  operation  might  work.  Or  require  students  to  explain  the  critical 
difference  between  two  contrasting  cases. 

3.  Use  of  worked  examples.  For  example,  provide  a  worked  example  of  how  to  solve  an  equation 
and  slowly  fade  away  the  amount  of  support  over  a  series  of  problems. 


Develop  measures 


A  set  of  measures  have  been  identified  for  the  research.  The  types  of  measures  are:  (a)  prior 
knowledge  of  the  content;  (b)  post-game  knowledge  of  the  content;  (c)  background  information 
such  as  prior  math  grades,  SAT/ACT  scores,  game  experience,  demographics  information  (age, 
sex,  race,  class  standing,  major);  (d)  English  language  proficiency;  and  (e)  math  self-efficacy 
and  interest  in  math.  Draft  measures  are  shown  in  Appendix  Pl-B  to  Appendix  . 


The  knowledge  measures  will  target  the  specific  content  covered  in  the  game  in  various  formats 
(e.g.,  cast  in  terms  of  the  game;  cast  symbolically  as  in  a  typical  math  test).  Items  susceptible  to 
common  errors  also  will  be  included  in  the  item  set,  as  well  as  items  that  require  students  to 
explain  their  problem  solving  step-by-step.  The  non-knowledge  items  will  be  adapted  from  prior 
studies. 


Develop  high-level  game  requirements 


A  preliminary  set  of  broad  requirements  for  the  game  was  developed  from  the  knowledge 
acquisition  sessions.  Specifically,  the  following  strategies  were  identified  as  desirable  for  the 
Math  90  student  population: 


•  Use  interrogation  strategies  typical  in  games — questioning  avatar  (vs.  telling  avatar) 

•  Embed  motivational  components  such  as  incremental  success  and  self-reflection 
strategies 

•  Focus  on  diagnostics — detect  what  students  know  and  common  errors 

•  Report  diagnostics  back  to  instructor  and  students 

•  Focus  on  type  of  instructional  feedback 

•  Questioning  vs.  telling — good  game  design 

•  Challenge  level — ok  for  “weird”  assessment  levels 

•  Focus  directly  on  symbolic  manipulation 


Phase  2 

During  this  phase  a  prototype  game  was  developed  and  pilot  tested  with  CSUSB  students.  Based 
on  student  responses,  the  prototype  was  updated  to  modify  the  type  of  feedback  provided  about 
specific  errors.  This  modified  version  was  also  class  tested  during  this  phase. 

The  prototype  game  consisted  of  22  levels.  See  Appendix  P2-A  for  a  screenshot  of  the 
prototype  version  of  the  game.  Levels  1-12  required  participants  to  select  the  immediate  next 


step  to  the  problem.  Levels  13-22  required  participants  to  solve  multistep  problems.  Participants 
were  shown  a  total  points  (one  point  for  each  correct  step  selected),  bonus  points  (one  point  for 
each  “mental  step”  step  elected),  and  total  completed  problems  (one  point  for  each  problem 
completed  before  time  ran  out). 

Based  on  the  pilot  test  performed  in  November  2012,  it  was  apparent  that  students  not  only 
enjoyed  playing  the  game,  but  also  felt  that  they  learned  from  the  game.  In  particular,  students 
reported  that  they  thought  that  receiving  feedback  immediately  on  each  mathematical  step  was 
more  effective  than  having  to  work  through  an  entire  math  problem  and  not  know  which  step  in 
the  process  might  have  contributed  to  an  incorrect  solution. 

However,  student  responses  related  to  the  next  step  process  (as  a  form  of  instruction)  and  the 
type  of  feedback  provided  (e.g.,  receiving  feedback  about  specific  errors)  seemed  somewhat 
unclear.  As  such  the  prototype  was  updated  to  provide  more  explicit  and  targeted  feedback 
options  related  to  each  mathematical  step,  and  more  sophisticated  help  options/menus  for 
students  to  access  when  experiencing  difficulty  during  gameplay.  Screenshots  depicting 
example  feedback  options  and  help  menus  are  provided  in  Appendix  P2-B. 

Additional  levels  were  also  created  to  reflect  more  consistency  across  math  item  expressions  and 
steps.  In  other  words,  representations  of  mathematical  expressions  or  transformations  were 
created  to  be  consistent  across  items  as  well  as  consistent  with  how  the  students  were  taught  to 
transform  math  items  and  depict  math  expressions.  The  following  section  provides  further  detail 
about  the  specific  instructional  variations  within  the  game. 

Develop  instructional  variations  within  the  game 

The  following  key  instructional  variation  components  were  developed  for  testing  purposes 
within  the  game: 

•  Step-to-step  transitions:  the  math  problems  were  structured  such  that  students  were  able 
to  observe  the  steps  required  to  solve  the  problem  including  incorrect  steps  and  common 
errors,  and  select  from  those  steps  the  next  correct  step  in  the  problem  sequence.  See 
Appendix  P2-C  for  an  example. 

•  Help  menu  with  worked  examples  and  explanations:  these  help  features  were  available 
for  students  to  access  when  needed  during  gameplay,  and  certain  feedback  prompts 
recommended  students  view  specified  help  menus  for  further  explanation.  (See  Appendix 
P2-B) 

•  Feedback  variations 

o  Implicit:  this  type  of  feedback  allowed  students  to  understand  that  the  step  they 
selected  was  either  incorrect  or  correct  by  either  sticking  to  or  not  sticking  to  the 
target  problem  or  preceding  step. 

o  Knowledge  of  results:  this  type  of  feedback  provided  students  with  an  image  or 
statement  indicating  that  their  action  was  correct  or  incorrect  (e.g.,  “Sorry,  that 
was  incorrect.  Please  try  again.”),  or  that  they  received  a  bonus  point. 


o  Elaboration:  this  type  of  feedback  provided  students  with  an  elaborated  statement 
and  suggestion  about  their  miss  step  (e.g.,  “Oops!  Looks  like  you  separated  the 
negative  and  you  shouldn’t  have.  It  should  stay  with  the  125  under  the  root  bar. 
Think  about  what  number  is  the  cube  root  of  125”).  A  variation  of  this  feedback 
type  also  included  an  image  of  an  example  problem,  or  directing  students  to  the 
help  menu  for  further  examples. 


Determine  overall  game  effectiveness  and  effectiveness  of  instructional  variations 

To  determine  overall  game  effectiveness  and  usability,  and  effectiveness  of  instructional 
variations,  three  tests  were  performed;  November  2012,  February  2013,  and  May  2013.  The 
below  sections  detail  the  November  2012  and  February  2013  studies.  The  data  for  the  May  2013 
study  are  being  analyzed. 

November  2012:  The  purpose  of  the  November  2012  testing  was  to  determine  game  usability, 
receive  student  feedback  on  the  game-playing  experience,  and  test  the  software  data  logging. 
Fifteen  participants  played  the  prototype  game  for  30-60  minutes.  After  playing  the  prototype 
game,  participants  were  administered  a  survey.  The  survey  consisted  of  24  selected  response 
questions  and  eight  open-ended  response  questions. 

Participants  reported  enjoying  the  experience  of  playing  the  prototype.  Participants  also  reported 
that  they  learned  from  the  game  and  that  the  game  helped  them  with  something  they  didn’t  know 
before.  They  thought  that  this  type  of  activity  (getting  feedback  immediately  on  each  step)  was 
more  effective  than  other  similar  activities  such  as  homework.  However,  when  asked 
specifically  about  the  next  step  process  and  receiving  feedback  about  errors,  there  was  a  wide 
range  of  responses  They  also  thought  that  earning  points,  especially  the  bonus  points,  was 
motivating. 

February  2013:  The  purpose  of  the  February  2013  testing  was  to  determine  game  effectiveness 
and  effectiveness  of  measures,  check  for  learning  gains  and  condition  differences  (i.e.,  feedback 
variations),  gather  student  perception  data  (i.e.,  math  attitudes  and  game  perceptions),  and 
determine  that  student  process  data  was  being  logged  correctly  during  gameplay. 

Participants  were  randomly  assigned  to  both  the  control  (n=  57)  and  treatment  (n=  54)  condition. 
Data  analysis  indicated  that  the  conditions  were  similar  across  most  background  and 
demographic  variables.  However,  the  control  group  reported  significantly  higher  ( p  =  .04) 
interest  in  math  than  the  treatment  group,  and  marginally  significant  differences  {p  =  .06)  were 
found  for  GPA  between  groups  with  the  treatment  group  scoring  higher.  No  differences  were 
found  in  pretest  scores  and  self-belief  in  math.  (See  Appendix  P2-D,  Table  1  for  details). 


An  ANOVA  was  conducted  to  determine  if  there  was  an  interaction  between  assessment  form 
and  condition.  No  significant  interaction  was  found  on  pretest  or  posttest,  and  no  main  effects 
for  assessment  form  or  condition  were  found  on  pretest  or  posttest.  Results  further  indicated  that 
were  was  no  significant  difference  between  conditions  on  the  posttest. 

Analysis  of  performance  on  math  scores  revealed  that  there  was  an  approximately  1 1  %  gain 
from  pretest  to  posttest  for  both  the  control  and  treatment  condition.  (See  Appendix  P2-D,  Table 
2  for  details).  Table  3  (see  Appendix  P2-D)  reports  item-level  performance  on  the  pretest  and 
posttest.  These  findings  indicated  that  approximately  12%  of  students  scored  incorrect  on  two 
pretest  items  then  correct  on  the  posttest,  and  1 8%  increased  from  incorrect  to  correct  on  and 
additional  three  other  items  on  the  posttest.  The  remaining  two  items  showed  increases  between 
28%  and  40%  of  students  from  pretest  to  posttest.  Individual  test  items  were  also  analyzed  using 
the  McNemar  test  to  determine  significant  change  from  pretest  to  posttest.  Results  indicated  a 
significant  change  from  pretest  to  posttest  for  items  5,  10,  and  11.  (see  Table  3,  Appendix  P2-D). 
In  sum,  the  results  indicated  a  small  overall  increase  of  scores  from  pretest  to  posttest,  with 
higher  performance  gains  appearing  for  specific  items. 

Based  on  the  February  2013  testing,  a  few  more  levels  were  added  to  the  game  to  target 
additional  math  problems,  some  images  were  changed  to  more  clearly  depict  feedback  and 
examples,  and  some  items  on  the  pre  and  post  assessment  were  changed  based  on  instructor 
feedback  and  student  performance. 

May  2013:  The  main  purpose  of  the  May  2013  testing  was  to  evaluate  an  initial  version  of  the 
game  based  on  changes  after  the  February  results.  Similar  to  February  2013  testing,  the  goal  for 
May  2013  testing  was  also  to  determine  game  effectiveness  and  learning  gains,  check  for 
differences  across  conditions  (i.e.,  feedback,  no  feedback),  and  test  software  data  logging  and 
gather  student  perception  data  regarding  math  and  gameplay. 


Phase  3 

The  major  scientific  objectives  of  this  phase  were  to  determine  the  major  types  of 
mathematical  errors  committed  by  students  during  gameplay  and  to  redesign  the  game  levels  to 
provide  more  focused  practice  on  those  topics. 

Error  Analysis 

Prior  to  conducting  the  error  analysis,  the  problems  used  in  the  game  were  subjected  to  a 
feature  analysis.  The  feature  analysis  identified  unique  properties  of  the  problems  in  terms  of  the 
structure  of  the  problem  (e.g.,  whether  the  problem  has  negative  terms),  cognitive  demands  (e.g., 
whether  students  were  required  to  transform  an  expression  from  an  exponent  form  to  a  radical 


form),  and  mathematical  topic  (e.g.,  application  of  the  negative  exponent  rule).  Table  1  shows  the 
set  of  features  used  to  describe  the  problems  used  in  the  game. 


Table  1. 

Features  of  Problem  Steps  in  Game  (5/21/2013  game  version). 

Category  Definition 

Problem  type 

Exponent  to  exponent  A  term  is  transformed  from  an  exponential  term  to  an  exponential  term 

Exponent  to  root  A  term  is  transformed  from  an  exponential  term  to  a  radical  term 

Root  to  exponent  A  term  is  transformed  from  a  radical  term  to  an  exponential  term 

Root  to  root  A  term  is  transformed  from  a  radical  term  to  a  radical  term 

Term  properties 

i 

XT  t  ^  The  number  of  distinct  terms  in  the  problem  (e.g.,  8x2  has  two  terms,  8  and 

Number  of  terms  ±  r  v  & 

X2) 

Term  type 

Letter  or  variable  Terms  represented  by  letters  (e.g.,  a ,  b ,  x,  y) 

Number  or  digit  Terms  represented  by  numbers  (e.g.,  1,  2,  3) 

Negative  terms 

Inside  parentheses  or  root  A  negative  term  has  a  negative  inside  the  parentheses  or  root  (e.g.,  (— 8)2) 

Outside  parentheses  or  root  A  negative  term  has  a  negative  outside  the  parentheses  or  root  (e.g.,  —  (8)2) 

Term  operations 

Addition  or  subtraction  Two  or  more  terms  are  added  or  subtracted  in  the  solution 

Multiplication  or  division  Two  or  more  terms  are  multiplied  or  divided  in  the  solution 

Fraction  properties 

1 

The  number  of  distinct  fractional  terms  in  the  problem  (e.g.,  2sx  has  one 
Number  of  fraction  terms  1  *  0 

fractional  term,  2s) 

Exponent  term  type 

Fraction  on  letter  or  3  5 

variable  Fractional  terms  appear  on  letter  type  terms  (e.g.,X2, 3/3) 

Fraction  on  number  or  digit  Fractional  terms  appear  on  number  type  terms  (e.g., 22, 4&) 

Negative  fraction  Negative  fractional  terms  in  the  problem  (e.g.,  x-2) 


Category 

Definition 

Non- trivial 

Non-1  numerator 

A  numerator  that  is  not  1 ; 
has  a  trivial  numerator) 

i 

if  1,  the  term  is  raised  to  the  1st  power  (e.g.,  ai 

Non-1  denominator 

A  denominator  that  is  not 
has  a  trivial  denominator) 

2 

1;  if  1,  the  exponential  term  is  an  integer  (e.g.,  oa 

Concept 

Zero  exponent  rule 

a°  =  1 

if  a  =£  0 

Negative  exponent  rule 

m  i  i  m 

a~n  =  —  and  — m  =  an 

an  a~~n 

if  a  =£  0 

Product  rule 

ar  ■  as=  ar+s 

Quotient  rule 

CL  y _ ^  1 

a  ~  apr 

if  a  =£  0 

Power  rule 

0 2r)S  =  ar's 

Note.  Names  and  definitions  taken  from  Math  90  textbook. 


Types  of  Errors 

The  problem  steps  associated  with  errors  committed  by  at  least  30  students  were  examined 
and  represented  25  different  errors.  Thirty  students  represented  30%  of  the  sample  and  the  types 
of  errors  appeared  to  be  representative  of  the  remaining  errors  in  the  sample.  The  errors  were 
committed  over  12  game  levels,  with  level  45  having  the  highest  number  of  errors  (7). 

Errors  were  detected  when  students  attempted  to  specify  the  next  step  given  an  existing  step. 

In  general,  the  next  step  represented  the  application  (or  misapplication)  of  an  exponent  rule  and 

inspection  of  the  errant  next  step  provided  insight  into  students’  understanding  as  students 

1  1 

attempted  to  simplify  the  expression.  For  example,  on  level  21,  given  the  expression,  37  ■  37,  58 
1  1 

students  specified  97  as  the  next  step  and  57  students  specified  37  as  the  next  step.  In  both  cases, 
the  misapplication  of  the  power  rule  is  clear  and  reflects  a  common  error  of  multiplying  exponents 
(instead  of  adding  them).  The  second  error  also  reflects  an  additional  misconception  of  multiplying 
the  base.  The  25  errors  committed  by  at  least  30  or  more  students  is  given  in  Appendix  P3-A  and 
frequency  information  on  all  errors  committed  by  all  students  is  given  in  Appendix  P3-B. 

To  better  understand  the  context  of  the  errors,  each  error  was  examined  in  terms  of  the 
concept  that  was  misapplied,  the  term  property,  and  problem  type  (as  defined  by  the  feature 
analysis  in  Table  1).  Many  of  the  errors  were  related  to  the  misapplication  of  the  power  rule, 
incorrect  transformation  between  exponent  and  root  forms,  apparent  confusion  about  the 
convention  related  to  the  negative  sign  and  powers,  and  the  misapplication  of  the  negative 


exponent  rule.  These  major  errors  served  as  the  basis  for  redesigning  the  game  levels  and 
assessments. 

Game  Level  Revisions 

The  revised  game  level  design  incorporates  the  results  of  the  error  analysis  in  the  following 
ways.  First,  the  problems  in  the  new  game  levels  focus  exclusively  on  the  following  concepts:  (a) 
exponent  to  root  transformations;  (b)  negative  exponent  rule;  and  (c)  power  rule.  Second,  the 
problems  incorporate  negative  numbers  both  as  exponent  properties  as  well  as  term  properties. 
Finally,  problems  sets  composed  of  4  levels  have  been  created  to  structure  gameplay  to  emphasize 
practice  and  reflection.  As  in  the  original  game,  the  game  levels  are  divided  into  2  stages.  Stage  1 
is  designed  to  provide  students  with  practice  on  basic  concepts  and  Stage  2  is  designed  to  provide 
students  with  applying  the  concepts.  Additionally,  two  types  of  problem  types  have  been  added:  a 
self-explanation  question  that  asks  students  to  explain  their  choice,  and  a  challenge  problem  that 
gives  students  only  a  single  chance  to  transform  the  problem.  The  challenge  problem  serves  as  an 
in-game  measure  of  learning  for  a  particular  problem  set. 

Stage  1:  Single  step  transformations.  Transformations  cover  basic  concepts  using  single¬ 
term,  single-step  problems.  There  are  six  problem  sets  (PS)  in  Stage  1.  PS  1  requires  students  to 
transform  from  exponential  form  to  root  form,  PS  2  requires  students  to  transform  negative 
exponents  into  positive  exponents,  and  PS  3  combines  these  two  concepts.  PS  4  provides  practice 
with  negative  signs.  PS  5  increases  complexity  of  expressions  by  including  negative  terms  and 
positive  and  negative  fractional  exponents.  PS  6  introduces  students  to  the  power  rule,  required  for 
multi-term  problems  in  Stage  2. 

Each  PS  includes  four  problems:  (a)  a  single-step  problem  with  feedback;  (b)  a  single-step 
problem  requiring  students  to  explain  their  reasoning  behind  their  answer;  (c)  another  single-step 
problem  with  feedback  to  gauge  any  change  in  their  reasoning  based  on  their  self-explanation;  and 
(d)  a  challenge  problem  with  no  feedback.  Table  2  shows  the  Stage  1  problem  set  sequence. 


Table  2. 

Stage  1  Transformations  Problem  Set  Sequence. 


Problem  set 

Term 

property 

Exponent  property 

Concepts 

la-d 

Positive 

Positive  fractions 

Exponent  to  root 

2a-d 

Positive 

Negative  integers 

Negative  exponent  rule 

3a-d 

Positive 

Negative  fractions 

Negative  exponent  rule,  exponent  to  root 

4a-d 

Negative 

Positive  integers 

Negative  sign 

5a-d 

Negative 

Positive  and  negative  fractions 

Exponent  to  root,  negative  exponent  rule 

6a-d 

Positive 

Positive  integers  and  fractions 

Power  rule 

Stage  2:  Multistep  transformations.  Stage  2  problem  sets  require  multistep  solutions.  PS 
7  includes  four  problems:  Problems  a  to  c  are  multi-step  problems  with  feedback  and  problem  d 
is  a  challenge  problem.  No  self-explanation  problems  are  included.  The  remaining  problem  sets 
require  some  use  of  the  power  rule  to  distribute  an  exponent  to  multiple  terms.  The  steps  also 
require  students  to  apply  the  rules  learned  in  the  Stage  1  problem  sets.  PS  8  to  1 1  include  three 
problems:  Problems  a  and  b  are  multi-step  problems  with  feedback  and  problem  c  is  a  challenge 
problem.  No  self-explanation  problems  are  included.  Table  3  shows  the  Stage  2  problem  set 
sequence. 


Table  3. 

Stage  2  Multistep  Transformations  Problem  Set  Sequence. 


Problem  set 

Term  property 

Exponent  property 

Concepts 

7a-d 

Positive 

Positive  and  negative  fractions 

Exponents  to  roots,  negative  exponent  rule 

8a-c 

Positive 

Positive 

Power  rule 

9a-c 

Positive 

Negative 

Power  rule,  negative  exponent  rule 

lOa-c 

Negative 

Positive 

Power  rule,  negative  sign 

lla-c 

Positive 

Negative 

Power  rule,  negative  exponent  rule 

Phase  4 


Game  goal  and  mechanics.  The  goal  of  the  game  is  for  players  to  simplify  a  given 
expression  by  specifying  the  next  step  in  the  simplification  process.  To  start  the  process,  a  single 
tile  with  a  picture  of  the  to-be-simplified  expression  is  placed  on  the  main  game  board.  Time 


pressure,  a  common  game  design  element,  is  incorporated  by  having  the  tile  slowly  creep  to  the 

top  of  the  game  board.  The  player  must  select  the  “next  step”  from  a  set  of  available  tiles  on  left 

side  of  the  game  board  and  drop  it  on  the  target  tile  as  shown  in  Figure  1.  If  the  dropped  tile  is  a 

correct  next  step,  the  dropped  tile  is  attached  to  the  bottom  of  the  target  tile.  In  Figure  1,  the  initial 

1 

tile  is  the  expression  (—125)3  and  a  valid  next  step  has  been  dropped  on  the  initial  tile.  The  stack 
is  slowly  moving  to  the  top  of  the  board. 

To  help  students  to  focus  on  the  micro  steps  between  the  transitions  between  major  steps,  the 
game  awards  bonus  points  when  students  use  a  micro  step,  with  a  “bonus”  splash  screen  appearing 
momentarily  to  indicate  a  bonus  (or  use  of  a  micro  step). 


Figure  7.  User  interface  of  exponent  game 

Instructional  design.  The  game  focuses  on  the  following  concepts:  (a)  exponent  to  root 
transformations;  (b)  negative  exponent  rule;  and  (c)  power  rule.  Many  of  the  problems  incorporate 
negative  numbers  both  as  exponent  properties  as  well  as  term  properties.  Problem  sets  composed 
of  4  levels  each  structure  gameplay  to  emphasize  practice  and  reflection.  The  game  levels  are 
divided  into  2  stages.  Stage  1  is  designed  to  provide  students  with  practice  on  basic  concepts  and 
Stage  2  is  designed  to  provide  students  with  applying  the  concepts.  Additionally,  challenge 
problems  exists  that  gives  students  only  a  single  chance  to  transform  a  problem.  The  challenge 
problem  also  serves  as  an  in-game  measure  of  learning. 

Stage  1  design.  There  were  24  levels  in  Stage  1 .  The  levels  cover  basic  concepts  using  single¬ 
term,  single-step  problems.  There  are  six  problem  sets  (PS)  in  Stage  1.  PS  1  requires  students  to 


transform  from  exponential  form  to  root  form,  PS  2  requires  students  to  transform  negative 
exponents  into  positive  exponents,  and  PS  3  combines  these  two  concepts.  PS  4  provides  practice 
with  negative  signs.  PS  5  increases  complexity  of  expressions  by  including  negative  terms  and 
positive  and  negative  fractional  exponents.  PS  6  introduces  students  to  the  power  rule,  required  for 
multi-term  problems  in  Stage  2. 

Each  problem  set  includes  four  problems:  (a)  a  single-step  problem  with  feedback;  (b)  a 
single-step  problem  requiring  students  to  explain  their  reasoning  behind  their  answer;  (c)  another 
single-step  problem  with  feedback  to  gauge  any  change  in  their  reasoning  based  on  their  self¬ 
explanation;  and  (d)  a  challenge  problem  with  no  feedback.  Table  4  shows  the  Stage  1  problem  set 
sequence. 

Table  4. 


Stage  1  Transformations  Problem  Set  Sequence. 


Problem  set 

Term 

property 

Exponent  property 

Concepts 

la-d 

Positive 

Positive  fractions 

Exponent  to  root 

2a-d 

Positive 

Negative  integers 

Negative  exponent  rule 

3a-d 

Positive 

Negative  fractions 

Negative  exponent  rule,  exponent  to  root 

4a-d 

Negative 

Positive  integers 

Negative  sign 

5a-d 

Negative 

Positive  and  negative  fractions 

Exponent  to  root,  negative  exponent  rule 

6a-d 

Positive 

Positive  integers  and  fractions 

Power  rule 

Stage  2  design.  There  are  16  levels  in  Stage  2.  The  levels  in  Stage  2  problem  sets  require 
multistep  solutions.  PS  7  includes  four  problems:  Problems  a  to  c  are  multi-step  problems  with 
feedback.  No  self-explanation  problems  are  included  in  Stage  2.  The  remaining  problem  sets 
require  some  use  of  the  power  rule  to  distribute  an  exponent  to  multiple  terms.  The  steps  also 
require  students  to  apply  the  rules  learned  in  the  Stage  1  problem  sets.  PS  8  to  1 1  include  three 
problems:  Problems  a  and  b  are  multi-step  problems  with  feedback  and  problem  c  is  a  challenge 
problem.  No  self-explanation  problems  are  included.  Table  3  shows  the  Stage  2  problem  set 
sequence. 


Table  5. 


Stage  2  Multistep  Transformations  Problem  Set  Sequence. 


Problem  set 

Term  property 

Exponent  property 

Concepts 

7a-d 

Positive 

Positive  and  negative  fractions 

Exponents  to  roots,  negative  exponent  rule 

8a-c 

Positive 

Positive 

Power  rule 

9a-c 

Positive 

Negative 

Power  rule,  negative  exponent  rule 

lOa-c 

Negative 

Positive 

Power  rule,  negative  sign 

lla-c 

Positive 

Negative 

Power  rule,  negative  exponent  rule 

Research  Questions 

The  main  research  questions  for  this  study  are:  To  what  extent  does  self-explanation  of 
exponent  rules  in  a  game  affect  their  application  of  exponent  rules  and  perceptions  of  engagement? 
To  what  extent  does  can  game-based  practice  of  exponent  rules  improve  students’  application  of 
those  exponent  rules? 

Method 

Participants  and  Design 

Participants.  The  sample  comprised  69  participants  (19  males,  47  females,  3  unreported) 
who  were  ethnically  diverse  (42  Latino/a,  9  White,  8  Black  or  African  American,  2  Asian  or  Pacific 
Islander,  3  Multiracial/multiethnic,  and  5  unspecified)  and  with  a  mean  age  of  19.2  years  old  ( SD 
=  3.8  years).  When  asked  how  frequently  a  language  other  than  English  is  spoken  at  home,  20 
students  (29%)  reported  all  or  most  of  the  time,  23  (33%)  about  half  of  the  time,  12  (17%)  once  in 
a  while,  and  1 1  (16%)  never.  Eighteen  students  reported  they  were  repeating  the  class,  28  students 
took  the  preceding  course,  and  20  students  reported  this  was  their  first  college  math  course.  The 
mean  overall  GPA  reported  by  students  was  3.16  (SD  =  0.47)  on  a  4.0  scale.  When  asked  about 
their  gameplay  skill,  16  students  (23%)  reported  they  were  very  good,  13  (19%)  reported  good,  24 
(35%)  reported  their  skill  as  fair,  and  13  (19%)  reported  poor  gameplay  skills.  When  asked  how 
many  hours  per  week  they  spend  playing  games,  26  participants  (38%)  reported  0  hours,  29 
participants  (42%)  reported  1-4  hours,  7  participants  reported  5-8  hours,  and  1  participant  report 
13  or  more  hours.  Finally,  the  mean  score  (on  a  1  to  4  point  scale)  for  participants’  interest  in  math 
and  math  self-efficacy  were  M=  2.48  (SD  =  0.71)  and  M=  2.5  (SD  =  0.75).  Overall,  the  sample 
appeared  to  be  of  average  achievement,  ethnically  diverse,  and  have  a  moderate  degree  of  game 
experience.  Participants’  interest  and  self-efficacy  were  at  the  scale  midpoints  and  normally 
distributed,  suggesting  the  sample  was  not  overly  interested  or  disinterested  in  math,  nor 
excessively  possess  high  or  low  in  math  self-efficacy. 


Design.  A  two-group,  pretest-posttest  experimental  design  was  used.  Participants  were 
randomly  assigned  to  either  a  baseline  game  condition  or  a  self-explanation  condition. 

Measures 

Prerequisite  knowledge.  Four  questions  were  asked  to  gather  information  on  participants’ 
proficiency  with  basic  fraction  addition,  multiplication,  and  order  of  operations. 

Rules  of  exponent.  Three  scales  were  developed  to  measure  three  skills:  (a)  converting  an 
exponent  to  its  root  form  (ETR),  (b)  application  of  the  negative  exponent  rule  (NER),  and  (c) 
application  of  the  power  rule  (PR).  A  fourth  scale  was  developed  to  examine  transfer  performance 
(XFER)  and  included  topics  from  each  of  the  other  scales  but  the  problems  were  more  complex  in 
terms  of  the  number  of  variables  in  the  expressoin.  The  ETR  scale  had  8  items  (pretest  a  =  .73; 
posttest  a  =  .74),  NER  scale  6  items  (pretest  a  =  .60;  posttest  a  =  .73),  the  PR  scale  had  4  items 
(pretest  a  =  .82;  posttest  a  =  .65),  and  the  XFER  scale  had  5  items  (posttest-only  a  =  .71). 

Game  perception.  Two  scales  were  used  to  measure  participants’  perception  of  their  game 
experience.  The  engagement  scale  (I  really  got  into  the  game,  Doing  the  game  was  boring,  I  would 
have  liked  to  play  longer,  I  thought  the  game  was  a  waste  of  time,  Beating  the  different  levels  made 
me  feel  good)  and  a  perceived  learning  scale  {I  learned  something  new  about  exponents/roots/math 
from  the  game,  The  game  helped  me  figure  out  something  I  previously  had  trouble  with,  I  realized 
I  had  a  misconception  about  an  exponent/root  topic  while  playing  the  game).  The  5-item  perceived 
engagement  scale  a  =  .75  and  3-item  perceived  learning  scale  a  =  .85. 

Feedback  about  the  game.  Participants  were  asked  to  provide  feedback  about  their 
experience  with  the  game.  Participants  were  asked  three  questions  ( How  do  you  think  this  game 
compares  to  other  ways  of  practicing  math  (e.g.,  doing  problems  on  paper)?;  How  helpful  (for 
learning)  do  you  think  it  was  to  emphasize  the  “next  step  ”  in  the  problem  solving  process?  Overall, 
how  much  did  you  learn  about  exponents/roots  from  the  game?).  Each  question  had  Likert-type 
anchors  as  well  as  space  for  written  comments. 

Background.  Participants  were  asked  to  complete  a  questionnaire  asking  for  their  age, 
gender,  ethnicity,  and  frequency  of  speaking  a  language  other  than  English  at  home.  In  addition, 
participants  self-reported  their  overall  GPA,  transfer  status,  skill  with  video  games,  and  number  of 
hours  per  week  spent  playing  games.  Participants  were  also  asked  about  their  interest  in  math  and 
math  self-efficacy.  These  scales  were  adopted  from  Marsh,  Hau,  Artlet,  Baumert,  and  Peschar 
(2006).  The  interest  in  math  scale  had  3  items  (a  =  .72)  and  math  self-efficacy  scale  3  items  (a  = 
.80). 


Game 


Base  version.  The  base  version  was  the  game  as  described  earlier.  There  were  24  levels 
requiring  single-step  transformations  (Stage  1)  and  16  levels  require  multiple-step  transformations 
(Stage  2).  Students  were  not  required  to  self-explain  their  selections. 

Experimental  version.  The  experimental  version  of  the  game  required  participants  to 
engage  in  self-explanation  during  selected  levels  in  Stage  1 .  There  were  6  levels  that  required  self- 
explanation.  There  were  4  levels  in  a  problem  set.  The  design  of  the  levels  within  a  problem  set 
was  to  first  provide  a  level  that  provided  minimal  feedback  to  participants  if  they  selected  an 
incorrect  step.  Then  a  level  with  a  similar  problem  was  presented  except  with  a  self-explanation 
prompt  appearing  immediately  after  the  student  dropped  the  next  step  tile  onto  target  tile  on  the 
game  board.  No  feedback  was  given  to  students  about  the  accuracy  of  their  self-explanation.  The 
self-explanation  prompt  was  a  list  of  exponent  rules.  Participants  were  instructed  to  choose  the 
rule  that  justified  their  next  step  as  shown  in  Figure  2. 


EXPONENT  RULES 

Zero-Exponent  a0  =  1(  if fl  *  0 

Rule  ’ 


Negative 
Exponent  Rule 


1 

»  =  — ,  if  a  0 


ar  1 

Quotient  Rule  »  —  —  ar  J  -  - ,  if  a  ^  0 

as  as_r 


Power  Rule  •  (ar  )s  -  ar  s 
Product  to 


Power  Rule 

Quotient  to 
Power  Rule 


*  (a,b)r=  aTJf 


Quotient  to  a  -r  //Ar 
Negative  Power  ft  ( — )  —  if  a 

Rule  W 


*  0 


Please  provide  a  justification  for  your 
step.  Click  on  a  round  button  next  to  the 
rule  that  best  explains  your  step. 

Step  l  is  the  step  on  the  game  board. 
Step  2  is  the  step  you  just  dropped. 


STEP  i 


STEP  2 
(the  step 
you  just 
added) 


DIRECTIONS 


OK 


CANCEL 


Figure  2.  Self-explanation  prompt 


Procedure 

Participants  were  introduced  to  the  study  and  then  provided  all  materials  (pretest,  laptop 
computer,  and  posttest).  Participants  were  instructed  to  start  on  the  pretest.  After  completing  the 
pretest,  the  pretest  was  collected  and  participants  were  instructed  to  start  the  game  on  the  laptop. 


After  completing  the  game  at  their  own  pace,  participants  started  the  posttest  and  completed  the 
test  at  their  own  pace.  All  participants  except  2  finished  all  materials.  These  two  participants  did 
not  finish  the  game  nor  the  posttest. 

Results 


Preliminary  Analyses 

A  check  for  differences  between  conditions  was  conducted  for  the  pretest  and  background 
measures.  An  independent  t-test  was  conducted  on  the  prerequisite  knowledge  items,  the  three 
rules  of  exponent  scales,  the  two  attitudinal  scales,  and  the  two  game  experience  questions.  There 
were  no  differences  between  conditions  on  any  measure  {p  >  .05)  suggesting  that  the 
randomization  procedure  worked. 


Descriptive  Statistics 


Table  6 


Descriptive  Statistics  for  Math  Scales  by  Condition  and  Occasion 


Total 

Self-explanation 

Control 

Scale 

N 

M 

SD 

n 

M 

SD 

n 

M 

SD 

Pre-requisite  knowledge"* 

2  1 

Multiplying  two  fraction:  -  X  - 

69 

.74 

.44 

33 

.16 

.44 

36 

.72 

.45 

Negative  sign  and  order  of  operations: 

— (— 2)2 

69 

.75 

.43 

33 

.82 

.39 

36 

.69 

.47 

2  3 

Adding  a  two  fractions:  -  +  — 

69 

.83 

.38 

33 

.91 

.29 

36 

.75 

.44 

Negative  sign  and  order  of  operations: 

i 

69 

.12 

.32 

33 

.06 

.24 

36 

.17 

.38 

-16? 


Converting  an  exponent  to  its  root  form  (ETR)6 


Pretest 

69 

3.00 

1.77 

33 

3.33 

1.57 

36 

2.69 

1.91 

Posttest 

67 

4.33 

1.93 

31 

4.39 

1.65 

36 

4.28 

2.16 

Applying  the  negative  exponent  rule  (NER)C 

Pretest 

69 

2.55 

1.57 

33 

2.21 

1.32 

36 

2.86 

1.73 

Posttest 

67 

3.27 

1.85 

31 

3.16 

1.59 

36 

3.36 

2.06 

Applying  the  power  rule  (PR)"* 

Pretest 

69 

2.49 

1.56 

33 

2.45 

1.56 

36 

2.53 

1.58 

Posttest 

67 

3.15 

1.12 

31 

3.32 

1.05 

36 

3.00 

1.17 

Transfer  (XFER)e 

67 

2.06 

1.52 

31 

2.19 

1.42 

36 

1.94 

1.60 

Note.  "Min.  =  0,  Max.  =  1.  Min.  =  0,  Max.  =  8. 

cMin. 

-  0,  Max. 

=  6.  Min.  - 

0,  Max.  - 

4.  Min. 

-  0,  Max. 

=  5. 

Main  Analyses 

Performance.  Separate  repeated-measures  ANOVAs  were  conducted  on  the  ETR,  NER,  and 
PR  scales.  To  simultaneously  determine  if  there  was  a  condition  effect  as  well  as  a  learning  effect, 
participants  performance  were  analyzed  with  a  2(condition)  x  2(occasion)  ANOVA,  with 
occasion  as  the  within- subjects  factors,  and  condition  as  the  between-subjects  factor.  No 
significant  condition  x  occasion  interaction  was  found  for  ETR,  NER,  or  PR  scales,  suggesting 
that  there  were  no  effects  of  self-explanation  on  participants’  learning  of  those  topics.  For  the 
transfer  task,  an  independent  /-test  was  conducted  on  the  XFER  scale  to  test  for  differences 
between  conditions.  No  differences  were  found. 


However,  a  significant  within-subjects  main  effect  of  occasion  was  found  for  ETR  (F[  1 ,  65] 
=  12.82,  MSE  =  1.12,/?  =  .001),  NER(F[1,  65]  =  19.42,  MSE  =  17.16,  p<  .001),  and  PR  (F[l, 
65]  =  12.82,  MSE  =  14.13,/?  =  .001),  suggesting  that  participants  learned  over  the  course  of  the 
game.  Table  7  displays  the  descriptive  statistics,  effect  size  of  the  change  (Dunlap,  Cortina, 
Vaslow,  &  Burke,  1996),  and  percent  increase  for  each  scale.  The  largest  increase  was  found  for 
the  exponent  to  root  transformation  problems. 

Table  7. 


Learning  Effects  of  the  Game  by  Topic 


Pretest 

Posttest 

% 

increase 

Scale 

n 

M 

SD 

M 

SD 

d 

Converting  an  exponent  to  its  root  form  (ETR)fl 

67 

3.00 

1.72 

4.33 

1.93 

0.73 

44.3 

Applying  the  negative  exponent  rule  (NER)6 

67 

2.57 

1.58 

3.27 

1.85 

0.40 

27.3 

Applying  the  power  rule  (PR)C 

67 

2.51 

1.54 

3.15 

1.12 

0.47 

25.6 

Note.  aMin.  =  0,  Max.  =  8;  ^Min.  =  0,  Max.  =  6; 

cMin. 

-  0,  Max. 

=  4. 

Perceived  learning  and  engagement.  To  examine  the  effects  of  self-explanation  on 
participants’  perceived  engagement,  an  independent  /-test  was  conducted  to  test  the  effects  of 
condition  on  perceived  learning  and  participants’  perceived  engagement.  No  differences  were 
found  for  participants’  perceived  learning.  However,  participants  who  received  the  self- 
explanation  prompts  ( M  =  3.07  SD  =  0.56),  compared  to  participants  who  did  not  receive  self- 
explanation  prompts  (M  =  2.68,  SD  =  0.58),  had  significantly  lower  ratings  of  their  experience, 
t(6 3)  =  2.73,  p  =  .008,  d  =  0.68.  The  large  effect  size  suggests  that  the  self-explanation  prompts 
interfered  with  participants’  perceived  engagement  with  the  game. 

Usability.  The  last  analysis  examined  participants’  perceptions  of  their  experience.  Table  8 
shows  the  frequency  of  responses  to  three  questions:  (a)  How  do  you  think  this  game  compares  to 
other  ways  of  practicing  math  (e.g.,  doing  problems  on  paper)?  (b)  How  helpful  (for  learning)  do 
you  think  it  was  to  emphasize  the  “next  step”  in  the  problem  solving  process?  and  (c)  Overall,  how 
much  did  you  learn  about  exponents/roots  from  the  game? 


Table  8. 

Participants’  Utility  Ratings  (N  =  64) 


Question 

Response  Anchors 

How  do  you  think  this  game  compares  to 
other  ways  of  practicing  math  (e.g., 
doing  problems  on  paper)? 

Much  less 
effective 

Less  effective 

Similar 

More 

effective 

Much  more 
effective 

3 

7 

16 

27 

11 

How  helpful  (for  learning)  do  you  think 
it  was  to  emphasize  the  “next  step”  in  the 
problem  solving  process? 

Extremely 

helpful 

Very  helpful 

Helpful 

Somewhat 

helpful 

Not  helpful  at 
all 

11 

10 

25 

17 

1 

Overall,  how  much  did  you  learn  about 
exponents/roots  from  the  game? 

A  lot 

Some 

Not  much  at 
all 

17 

43 

4 

Overall,  participants  reported  very  favorable  responses  to  the  game,  with  59%  of  the  same 
reporting  the  game  was  more  or  much  more  effective  than  other  ways  of  practicing  math.  This  is 
remarkable  considering  the  game  was  in  a  pre-prototype  stage,  did  not  mask  the  mathematics,  and 
provided  minimal  game  garnishes.  Thirty-three  percent  of  students  reported  the  emphasis  of  the 
next  step  as  being  very  or  extremely  helpful  for  learning,  with  an  additional  39%  reporting  the 
emphasis  on  “next  steps”  as  helpful.  Finally,  27%  of  students  reported  the  game  as  helping  them 
learn  a  lot  about  exponents/roots  and  67%  reporting  the  game  help  them  some. 

Conclusion 

Overall,  the  pre -prototype  game  appeared  to  provide  students  with  an  engaging  and 
beneficial  experience.  With  less  than  40  minutes  of  gameplay  time,  students  improved  their 
performance  on  math  problems  related  to  converting  an  exponent  to  its  root  form,  applying  the 
negative  exponent  rule,  and  applying  the  power  rule,  and  reported  high  levels  of  utility.  This  is  an 
interesting  finding  because  it  suggests  that  the  instructional  sequence,  game  mechanics,  and  focus 
on  problems  susceptible  to  common  errors  collectively  helped  students  improve  their  performance 
substantially,  as  indicated  by  the  moderate  to  large  pretest-to-posttest  effects  sizes  ranging  from 
.40  to  .73. 

The  lack  of  finding  an  effect  of  the  self-explanation  prompts  is  curious.  We  followed  the 
similar  procedures  used  by  Mayer  and  Johnson  (2010)  and  Johnson  and  Mayer  (2013),  who  found 
that  the  when  people  are  asked  to  select  self-explanations  for  their  moves  in  a  game,  they  learn 
more  (compared  to  people  who  are  not  asked  to  select  an  explanation).  Our  results  are  more  in  line 
with  O’Neil  et  al.  (2014),  who  found  no  main  effect  of  self-explanation  in  math  learning  in  a  game. 


O’Neil  et  al.  concluded  that  the  most  effective  self-explanations  where  those  that  prompted 
connections  between  the  concrete  elements  in  the  game  and  the  abstract  mathematics  terminology. 
The  least  effective  self-explanation  prompts  were  those  that  asked  very  simple  or  very  abstract 
questions.  It  may  be  the  case  for  our  game  and  participants,  the  self-explanation  prompts  may  been 
too  abstract  as  the  choices  were  the  symbolic  representation  of  the  exponent  rule.  Further  research 
is  needed  to  explore  this  issue. 
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Appendix  Pl-A 

Common  Student  Errors  and  Difficulties 


Type 


Examples 


Fundamentals 


Poor  understanding  of  grouping  symbols  (e.g.,  parentheses),  order  of 
operations,  fractions,  distribution,  equality.  Examples  of  rare  but  illustrative 
common  errors: 


(x  +  y)2  x2  +  y2  Distributing  the  exponent. 


V2  -2 
V3  3 


Canceling  the  radical. 


Canceling  the  2. 


2 


Canceling  the  2. 


Adding  the  numerators  (and  denominators) 
together  (i.e.,  across  the  fraction  bar). 


Notation 


Unfamiliarity  with  symbols  and  their  meanings 


Radicals  and 
exponents  (laws  of 
exponents) 


Negative  numbers  raised  to  some  power 
are  perceived  as  difficult,  with  students 
perceiving  the  expression  as  being  too 
complicated  or  too  many  things  to  do. 


Vie* 


Difficulty  decomposing  or  breaking  the 
problem  into  sub-parts. 


-16i 


Difficulty  remembering  the  convention 

1 

(i.e.,  (—16)4  )  and  will  interpret  the 

i 

expression  as  — (I64). 


11  — 

— t  same  as  — -  same  as  —8s  Misapplication  of  laws  of  exponents. 

8  “3  -83 


Type 

Motivation 


Problem  solving 


Examples 

Students  often  will  ask  instructors  to  just  tell  them  how  to  solve  the  problem. 
Students  often  lack  confidence  in  their  capabilities  to  learn  math.  Students 
also  appear  to  respond  positively  to  instructor  feedback  (e.g.,  use  of  "smiley" 
faces  to  indicate  good  job). 

Often  needs  help  with  the  first  step.  Students  often  lack  the  skills  in 
strategizing  about  the  problem  solving  process.  Students  will  often  appear  to 
make  up  their  own  rules,  ignore  rules,  confuse  rules. 


Appendix  Pl-B 
Background  Information 


GENERAL  INFORMATION 

1 .  Age: _ years 

2.  Gender:  □  Male  □  Female 

3.  Ethnicity: 


□  Biracial/multiethnic 

□  African  American 

□  Asian  American 

□  Latino 


□  Native  American 

□  White,  non-Hispanic 

□  Other _ 


4.  Academic  standing:  □  Freshman  □Sophomore  □  Junior  □  Senior  □  Other 


5.  Overall  GPA: 


6. 


Major  GPA: 


7. 

SAT  1  Verbal  Score: 

□  N/A 

8. 

ACT  Verbal  Score: 

□  N/A 

9. 

SAT  1  Math  Score: 

□  N/A 

10. 

ACT  Math  Score: 

□  N/A 

Appendix  Pl-C 

Language  Background  Information 


1 .  How  good  are  you  at  reading  English? 

□  Very  good  □  Good  □  Average  □  Poor 


2.  Do  you  speak  a  language  besides  English? 
□  Yes  □  No 


If  NO,  skip  to  the  page  titled  <PAGE  TITLE  HERE> 


If  YES,  what  is  that  language? _ 

If  YES,  please  fill  out  the  next  page. 


Always 

Most  of 

the  time 

Some¬ 

times 

Never 

3.  How  much  do  you  speak  that  language  with 

your  parents? 

□ 

□ 

□ 

□ 

4.  How  much  do  you  speak  that  language  with 

your  friends  at  school  and  outside  of  school? 

□ 

a 

□ 

a 

Very 

well 

Moder¬ 

ately 

well 

Fairly 

well 

Not  very 
well 

5. 

Do  you  speak  that  language  well? 

□ 

□ 

□ 

□ 

6. 

Do  you  understand  that  language  well? 

□ 

a 

□ 

□ 

7. 

Do  you  read  that  language  well? 

□ 

□ 

□ 

□ 

8. 

Do  you  write  that  language  well? 

□ 

□ 

□ 

□ 

9. 

Do  you  understand  spoken  English  well? 

□ 

□ 

□ 

□ 

10. 

Do  you  speak  English  well? 

□ 

□ 

□ 

□ 

11. 

Do  you  read  English  well? 

□ 

□ 

□ 

□ 

12. 

Do  you  write  English  well? 

□ 

a 

□ 

□ 

1 3.  In  which  language  do  you  read  better? 

□  English  □  Other  language 

14.  If  you  have  homework  that  you  don’t  understand,  and  you  need  to  ask  a  friend  about  it,  what 
language  would  you  prefer  to  use? 

□  English  □  Other  language 


Appendix  Pl-D 

Perceptions  of  Math  Questionnaire 


Disagree 

Disagree 

somewhat 

Agree 

somewhat 

Agree 

1 .  1  get  good  marks  in  mathematics. 

□ 

□ 

□ 

□ 

2.  Mathematics  is  one  of  my  best  subjects. 

□ 

□ 

□ 

□ 

3.  1  have  always  done  well  in  mathematics. 

a 

a 

a 

a 

4.  When  1  do  mathematics,  1  sometimes  get  totally 

□ 

□ 

□ 

□ 

absorbed. 

5.  Because  doing  mathematics  is  fun,  1  wouldn’t  want 

□ 

□ 

□ 

□ 

to  give  it  up. 

6.  Mathematics  is  important  to  me  personally. 

a 

a 

a 

a 

Marsh,  H.  W.,  Hau,  K-T.,  Artelt,  C.,  Baumert,  J.,  &  Peschar,  J.  L.  (2006).  OECD's  brief  self-report  measure 
of  educational  psychology's  most  useful  affective  constructs:  Cross-cultural,  psychometric  comparisons 
across  25  countries.  International  Journal  of  Testing,  6,  311-360. 


Appendix  Pl-E 

Example  Types  of  Math  Problems 


nth  Roots 


V9  +  VI6  z! 

(  xy\  (6a)  “4 

\xy-2J 


V64 


3 


-V(-3)6 


Exponents 


V6 

w 


_5  1 

9  4  X  93 


(64m2n 
m~2n3 


1 

2 


f 27x2y  1 
\  y"3x"2 


1  4N 
4x3y9 
1  2 
v*  3y3, 


1 

3 


Appendix  P2-A 

Example  Screenshot  of  Prototype 


Pre-prototype  Came 


ERROR 


2 


(-0025)’ 


Correct  step:  +  1  point 
Error  step:  - 1  point 
Micro-step:  Bonus  point 


FINAL 


MICRO 


ERROR 


MICRO 


ERROR 


CORR 


ERROR 

PsL,* 


[ 


r 


V  — 125 


(— 5)(  -5) 


-25 


(-5)* 


Game  mechanic: 
Drag  one  step  at  a 


Level  40 


Points 


Directions 
Exponent  Rules 


Example  Problems 


Navigation 

Play  Previous  Level 


Reset  Level 


Next  Level 


.v6 


Appendix  P2-B 
Feedback  and  Help 

Help:  Negative  integers 


Two  examples  with 


negative  integers 


Step  1 : 

-27 3 

Step  2: 

(-1)27* 

Step  3: 

(-1)27” 

Step  4 

(-l)(4/27) 

(-27)1 

U2 

(-27)5  T 

(ifZij 

W(-3)(-3)(-3)f 


Hints: 

Look  at  the 
parentheses  Are 
they  there?  Where 
are  they  placed? 

Compare  these  two 
examples  to  help 
you  figure  out  what 
to  do  with 
parentheses  and 
negative  numbers 


Steps:  (-1)(V(3)(3)(3))‘ 

Step  6:  (— 1)  (3)2 

Step  7:  (-1)9 


(-3)2 
(-3)  (-3) 
9 


Step  8' 


-9 


Help:  Positive  exponents 


Two  examples  with  exponents 


Step  1 


Step  2: 


r 

f 


x 


/  2  \  T 

x  y  3 

,  *y~2  j 


'xV 


Hints: 

Use  the  Laws  of 
Exponents  (button  in 
the  Help  menu)  to 
figure  out  how  to 
multiply,  divide,  add, 
and  subtract  rational 
exponents 


Step  3: 


x  2  y 3  N 


(*■>•’ )r 


Step  4:  (-V.V  ‘  ) 

Step  5:  (,vy 3  )  (xy 3  )  ( xy  3  ) 

Step  6:  (x  •  x  ■  x  •  y 3  ■  y 3  ■  y  '  ) 
Step  7:  x 3  y 9 


1  £_  1 
x  3  y  »  3 

i  1 
x  3  y  3 

x 3  y 


Help:  Negative  exponents 


Exponent  Laws 


Exponent  laws - - 

Law 

Example 

X 

II 

H 

X 

61  -  6 

X°  =  1 

7°  =  I 

x'1  =  1/x 

4‘1  =  1/4 

xmxn  _  xm  +  n 

x2x3  =  x2*3  =  X3 

xm/xn  =  xm'n 

xVx2  =  x*'2  -  x4 

(xm)n  =  xmn 

(x2)3  =  X2*3  =  x€ 

(xy)n  =  xnyn 

(xy)3  =  x3y3 

(x/y)n  =  xn/yn 

(x/y)*  =  xa  /  ys 

x'n  =  l/xn 

x'3  =  1/x3 

And  the  law  about  Fractional  Exponents: 

X **  —  \/x™ 

*3  =  <Hfl 

= 

Feedback  Images 


Congratulations,  you  completed 
the  game' _ 


Example  Feedback  in  Game:  Negative  Exponent  rule 


Negative  Exponent  Rule 


a  17  = 


a 


m 

77 


a*  0  and  '<[a  is  defined 


Exponent  is  negative 


Exponent  is  positive 


Watch  out!  A 
negative  in  the 
exponent  (-1/3) 
does  not  change 
die  sign  of  the 
base  (7)! 

7~5  *  (-7F 


Example  Feedback  in  Game:  nth  roots 


Change  the  fractional  exponent  to  an 
nth  root  expression, 

m  UL 

(zfa'}  =  a  "  = 

{MT)  =  iAj  = 


m 


a  prodded  exists 


ill 


Appendix  P2-C 

Example  of  Instructional  Variation 


Focus  on  Step-to-step  Transitions 

Micro-  Typical  Common 

steps  steps  errors 


(-125)72/3 

(V3&-125J72 
(V3&(-5)(-5)(-5))t2 


(-5)T2 


( —  1)  (125)  72/3 

/-125  73 
—  125  T2 


(-  5/r-s; 

25 


-25 


Appendix  P2-D 
Data  Analysis 


Tabic  I 

Awiadc*  Towardu  Mjcb  by  CowSiiiaa 


Loattol 

Treatment 

ln-54| 

4 

i 

Pretest 

4.'4i2.00i 

4.98  i_2_2B> 

109 

60 

imots  Jn  irwA 

1-52  (.66) 

2.2ft  (49J 

IDS 

imw 

^clF-belitl'  tn  nalb 

2JM  (T?p 

108 

,70 

CrPA 

IV)  (,7S) 

1  1.25  tii) 

n 

UlS  ^  eel:  vLdevgaiiire 

2J07(1.(T3 

2.37{3JDI} 

109 

Jt 

Skill 1  ■-  ClHiirce-K 

2.4ft(.95l 

im 

53 

^<-05. 


Table  ^ 

^  Malt  AchLcvctniait  by  Condition 


Number  of 
Items 

Control 
(ji=  57) 

Tieatnent 

(n=5A) 

loial 

(h=1»} 

Pretest  Cone 

7 

2.39(1.41) 

2.41(1.67) 

241  (1.53) 

Pomest  Cure 

7 

3.16(1.75) 

3.26(1.73) 

3.21  (E.73) 

Pretest- posttest  Cone  change 

.77 

.82 

.80 

Pen:  cp.i  i  nipiovement 

JJ% 

12% 

Lilt, 

IiMe  3 


:l:m  Arui/Mii- 


ff*L  L  Ll—BE 

c 

% 

T'  T-7A.-.  Z  i  1 

a 

*4 

[>adU>E  ~ r ■  i ■■ 

MEEMUEn-Jv; 

zhui^e 

i|lnqu-EDEYj 

Siculnl 
■  lEEMC_jn:yh 

Item  f 

]iMjiaim± 

V) 

bl 

46 

41 

45 

2 

fc-4+* 

CarrccB 

22 

S5 

E4 

Ifcm  b 

In-jnrrEEl 

VI 

M 

VI 

15 

13 

13 

S7 

■Harrr-:! 

7J 

f-4 

EJ 

llEni  ? 

]nCDfT€Et 

47 

42 

L-] 

47 

M 

*5 

Ed 

C  a  rrc-i  1 

E4 

!f;l 

Efl 

S3 

[;Em  P 

llWPIKE^ 

E? 

SI 

Ed 

Efl 

1  '3 

16 

T4 

Carre:! 

f4 

4'3 

55 

Si 

!km  'I1 

In-jnrTEEL 

Td 

■yJ 

S5 

£■1 

ia 

1 

B4 

C-Z-TTTill 

15 

3-1 

4* 

41 

Jeeeti  1  0 

Incnrr-EEi. 

5? 

57 

tfl 

EJ 

13 

4 

Cancel 

H 

13 

43 

16 

hrm  1  1" 

IrrCHfr-EE*. 

104 

rJ3 

*3 

M 

14 

4 

*3* 

Cancel 

d 

17 

IS 

xtuinti  mining 

*  p  <05,  “  p^.01 


Appendix  P3-A: 

Analysis  of  the  Errors  Committed  by  30  or  More  Students 


Image  ID 

(given/ 
next  step) 


Given  Step 


Next  Step 


Concept  Term  Property 


301/351 
1  (n  =  61) 


3 

-165-^32 


(ipU 


Negative  Terms 
Outside  Parentheses/Root 


3101/3151 
(n  =  58) 


\_  l 

32-32 


Product  Rule 


Term  Operations 
Mult/Div 


3101/3152 
(n  =  57) 


\_  l 

32-32 


Product  Rule 


303/351  i  — 1 1 1  ]  +!-l'l</32 

(n  =  54) 


Negative  Terms 
Outside  Parentheses/Root 


301/354 
(n  =  48) 


3 

-165-^32 


(<pL6)4-5/32 


Negative  Terms 
Outside  Parentheses/Root 


103/150 

6 

(  1  ^ 

42 

2\ 
a 2 

1  ^ 

1 

-c 

4 

f  2  1\ 

a-  2 

f  5  M 
b~~2  2 

II 

UJ 

\  J 

J 

J 

\  J 

\  J 

Exp  Term  Type 
On  Integer 


303/352 
(n  =  43) 


I  -1 1 1  \/l6  (  -1-1  j  \/32 


7 


8 


303/354 
(n  =  43) 


IV  *V 

4 2  tf2 

A  yv 


2a 

_3 


Negative 
Exponent  Rule 


4801/4852 

(n  =  42) 

r 

i  > 

1 

2  1 

9 

(27  bf  + 

4  b1 

(^27 

Power  Rule 

y 

) 

1403/1452  2  _2  _[ 

(n=42)  x3^-1^  3J3 


2  1 

x3  y2 


x\yx 


Power  Rule 


4502/4552 
(n  =  42) 


/- 1 — v 

r  s+n 

(V27  j 

r  3 

V  J 

Power  Rule 


2201/2204 
(n  =  41) 


2  V 

x_y 

-2 

y  j 


4801/4851 
(n  =  41) 


f+ 

f  IV 

4b 3 

V  J 

21b'  +Ab~3 


Power  Rule 


2203/2206 

14 

(n  =  40) 


9 


103/151 
(n  =  37) 


IV  2V 

4 2  a 2 


3 

bA 


V  y\ 


(  iV  l\C 
42  a2 


_4  > 

b~2 


v  yv  a  y 


No  Error: 
Missed  Steps 


No  Error: 
Missed  Steps 


Uncategorized  Error: 
Changed  Exponents 


15 


/ 


t 


4602/4652  /  .  1 

f  3  " 'l 

(n  =  37)  (9)2 

h^2 

V  J 

(9 ): 


j 

~  ~ 


l\ 


Power  Rule 


3501/3551 
(n  =  36) 


Simplifying 

Roots 


Term  Type 
Variable 


18 


301/352 
(n  =  35) 


3 

-165-^32 


1203/1253  _  ± 

(0-35)  (-1)92  ^273 


(-1 


'44T7\ 


3401/3451 
(n  =  34) 


Simplifying  Term  Type 

Roots  Variable 


1203/1254 

(n  =  33) 

(—1)9^ 

2 

+  273 

Negative  Terms 

Outside  Parentheses/Root 

303/302 

(n  =  32) 

(-111W1  ■ 

3 

(-1)16*  -H : -1)^32 

No  Error: 

Backwards  Step  Order 

1402/1451 

(n  =  31) 

2 

4y-‘  (x-' 

j( 

)3  y2  1 
l  J 

2  i  y 

V  J 

Power  Rule 

4301/4351 

24 

(n  =  31) 

1 

i 

9  1 

Q\ 

1 

Negative 

Exponent  Rule 

1202/1252 

25 

(n  =  30) 

3  2 

-9~2  +  2V 

+  (^27 ): 

Negative  Terms 

Outside  Parentheses/Root 

Appendix  P3-B: 

All  Errors  Committed  by  All  Students 


No  of 
Students 

Game 

level 

Image  ID 

No  of  attempts 

No  of  unique  attempts 

No  of  repeated  attempts 

Given  Step  Next  Step 

Min. 

Max. 

Min. 

Max. 

Min. 

Max. 

61 

45 

301 

351 

1 

2 

1 

1 

1 

2 

58 

21 

3101 

3151 

1 

2 

1 

1 

1 

2 

57 

21 

3101 

3152 

1 

3 

1 

2 

1 

1 

54 

45 

303 

351 

1 

4 

1 

2 

1 

2 

48 

45 

301 

354 

1 

6 

1 

3 

1 

3 

43 

44 

103 

150 

1 

6 

1 

2 

1 

3 

43 

45 

303 

352 

1 

6 

1 

3 

1 

2 

43 

45 

303 

354 

1 

9 

1 

5 

1 

3 

42 

44 

103 

152 

1 

9 

1 

4 

1 

3 

42 

48 

4801 

4852 

1 

4 

1 

2 

1 

2 

42 

49 

1403 

1452 

1 

4 

1 

3 

1 

2 

41 

41 

4502 

4552 

1 

4 

1 

2 

1 

2 

41 

43 

2201 

2204 

1 

2 

1 

1 

1 

2 

40 

48 

4801 

4851 

1 

2 

1 

1 

1 

2 

37 

43 

2203 

2206 

1 

2 

1 

2 

1 

1 

37 

44 

103 

151 

1 

8 

1 

3 

1 

2 

36 

42 

4602 

4652 

1 

4 

1 

2 

1 

2 

Image  ID 
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No  of  unique  attempts  No  of  repeated  attempts 


No  of  Game - 

Students  level  Given  Step  Next  Step  Min. 


35 

20 

3501 

3551 

1 

35 

45 

301 

352 

1 

34 

47 

1203 

1253 

1 

33 

19 

3401 

3451 

1 

32 

47 
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1254 

1 

31 

45 

303 

302 

1 

31 

49 
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1451 

1 

30 

38 

4301 

4351 

1 

30 

47 

1202 

1252 

1 

29 

49 
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1451 

1 

29 

49 

1402 

1452 

1 

29 

49 

1403 

1453 

1 

28 

47 

1203 

1252 

1 

28 

48 

4803 

4851 

1 

27 

40 

4401 

4451 

1 

27 

49 

1404 

1452 

1 

26 

34 

4201 

4251 

1 

26 

44 

102 

151 

1 

26 

44 

103 

153 

1 

26 

46 

2303 

2354 

1 

26 

47 

1201 

1252 

1 

26 

49 

1403 

1451 

1 

26 

49 

1404 

1453 

1 

25 

41 
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4552 

1 

25 

44 

101 

151 

1 

25 

48 

4802 

4852 

1 

24 

23 

3301 

3351 

1 

24 

42 

4603 

4652 

1 

24 

44 

103 

102 

1 

24 

48 

4802 

4851 

1 

23 

36 

1301 

1353 

1 

23 

42 

4601 

4652 

1 

23 

47 

1204 

1252 

1 

23 

49 

1406 

1454 

1 

Max.  Min.  Max.  Min. 


2  1 

4  1 

9  1 

3  1 

12  1 

2  1 

2  1 

2  1 

3  1 

2  1 

3  1 

6  1 

6  1 

2  1 

2  1 

4  1 

1  1 

3  1 

11  1 

7  1 

3  1 

4  1 

5  1 

3  1 

2  1 

4  1 

2  1 

6  1 

3  1 

1  1 

3  1 

3  1 

9  1 

4  1 


1  1 

2  1 

4  1 

1  1 

5  1 

1  1 

1  1 

1  1 

2  1 

1  1 

2  1 

4  1 

3  1 

1  1 

1  1 

3  1 

1  1 

2  1 

5  1 

6  1 

2  1 

2  1 

4  1 

2  1 

2  1 

2  1 

1  1 

3  1 

1  1 

1  1 

3  1 

2  1 

3  1 

4 


Max. 

2 

2 

3 

3 

3 

2 

2 

2 

2 

2 

1 

2 

2 

2 

2 

2 

1 

2 

2 

2 

2 

2 

3 

2 

1 

3 

2 

3 

3 
1 
2 
2 

4 


1 


1 


Image  ID 


No  of  attempts 


No  of  unique  attempts  No  of  repeated  attempts 


No  of  Game - 

Students  level  Given  Step  Next  Step  Min. 


23 

49 

1406 

1455 

1 

22 

36 

1303 

1353 

1 

22 

39 

1101 

1155 

1 

22 

41 

4501 

4551 

1 

22 

43 

2201 

2206 

1 

22 

47 

1201 

1251 

1 

22 

49 

1401 

1452 

1 

21 

36 

1301 

1352 

1 

21 

39 

1101 

1151 

1 

21 

41 

4503 

4552 

1 

21 

42 

4602 

4651 

1 

21 

44 

104 

153 

1 

21 

45 

302 

351 

1 

21 

47 

1201 

1254 

1 

21 

47 

1202 

1254 

1 

21 

47 

1204 

1254 

1 

20 

14 

1701 

1751 

1 

19 

45 

302 

354 

1 

19 

47 

1204 

1253 

1 

19 

48 

4803 

4852 

1 

19 

49 

1403 

1402 

1 

18 

11 

1501 

1551 

1 

18 

12 

2701 

2752 

1 

18 

41 

4504 

4551 

1 

18 

42 

4603 

4651 

1 

18 

44 

101 

150 

1 

18 

45 

302 

352 

1 

18 

47 

1202 

1253 

1 

18 

49 

1404 

1451 

1 

17 

34 

4203 

4253 

1 

17 

37 

204 

251 

1 

17 

37 

204 

252 

1 

17 

37 

204 

254 

1 

17 

41 

4502 

4551 

1 

Max.  Min.  Max.  Min. 


6  1 

4  1 

9  1 

2  1 

3  1 

2  1 

2  1 

2  1 

4  1 

3  1 

2  1 

6  1 

3  1 

6  1 

7  1 

15  1 

3  1 

7  1 

12  1 

2  1 

3  1 

1  1 

3  1 

2  1 

4  1 

1  1 

4  1 

5  1 

3  1 

4  1 

5  1 

7  1 

10  1 

2  1 


5  1 
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5  1 

1  1 

2  1 

1  1 

2  1 

2  1 

1  1 

2  1 

1  1 

5  1 

1  1 

4  1 

4  1 

5  1 

1  1 

4  1 

4  1 

2  1 

1  1 

1  1 

2  1 

1  1 

2  1 

1  1 

2  1 

3  1 

2  1 

3  1 

2  1 

3  1 

5  1 
1 


Max. 

2 

2 

2 

2 

1 

2 

2 

1 

4 
2 
2 
2 
3 
2 
2 
3 
3 
3 
3 
1 

3 
1 
2 
2 
2 
1 
2 
2 
1 
1 

5 

4 
2 


1 


2 


Image  ID 


No  of  attempts 


No  of  unique  attempts  No  of  repeated  attempts 


No  of  Game - 

Students  level  Given  Step  Next  Step  Min. 


17 

43 

2203 

2202 

1 

16 

35 

204 

254 

1 

16 

39 

1101 

1152 

1 

16 

39 

1102 

1152 

1 

16 

40 

4401 

4452 

1 

16 

40 

4402 

4452 

1 

16 

42 

4604 

4652 

1 

16 

43 

2201 

2252 

1 

16 

43 

2202 

2206 

1 

16 

43 

2203 

2251 

1 

16 

47 

1201 

1253 

1 

16 

47 

1204 

1255 

1 

16 

48 

4805 

4851 

1 

16 

49 

1403 

1454 

1 

15 

17 

2601 

2652 

1 

15 

44 

104 

152 

1 

14 

34 

4202 

4253 

1 

14 

36 

1303 

1302 

1 

14 

41 

4504 

4552 

1 

14 

44 

102 

150 

1 

14 

45 

303 
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1 

14 

47 

1204 

1251 

1 

13 

15 

1601 

1651 

1 

13 

22 
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3251 

1 

13 

35 

205 

254 
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13 

37 

204 

253 

1 

13 

40 

4402 

4451 

1 

13 

43 

2203 

2252 

1 

13 

47 

1203 

1202 

1 

13 

47 

1203 

1251 

1 

12 
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4152 

1 

12 

35 

204 

251 

1 

12 

35 

204 

252 

1 

12 

36 

1303 

1351 

1 

Max.  Min.  Max.  Min. 


2  1 

7  1 

6  1 

3  1 

4  1 

4  1 

3  1 

6  1 

2  1 

4  1 

4  1 

17  1 

1  1 

7  1 

2  1 

5  1 

3  1 

2  1 

3  1 

1  1 

7  1 

5  1 

1  1 

1  1 

3  1 

9  1 

2  1 

5  1 

3  1 

4  1 
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2  1 
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Max. 

2 
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1 
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level 
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1 

10 
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3 
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3 
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1 

2 

10 

48 

4804 

4851 

1 
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1 

1 

1 

1 

10 

49 

1405 
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1 
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1 
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1 

2 

9 

32 

4002 

4051 

1 

2 

1 

1 

1 

2 

9 

37 

204 
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1 

1 

1 

3 

9 

43 

2202 

2252 
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Max. 
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Min. 
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9 

45 

304 

353 

1 

4 

1 

4 

1 

2 

9 

46 

2302 

2354 

1 
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